Commit Graph

8082 Commits (73b2199fa28fa68bcf3c514f3b30da579af7c14b)

Author SHA1 Message Date
Matt Clay 73b2199fa2
[stable-2.8] [stable-2.9] [stable-2.10] Fix ansible-test handling of user-defined docker networks. (#72256) (#72310)
* Fix ansible-test docker container detection.

* Attach test containers to the correct network.

* Do not assume `localhost` for accesing Docker.

* Look for containers on current network.

* Always map /var/run/docker.sock into containers.

This fixes issues when using a remote Docker host.

* Support container IP lookup from networks list.

* Fix container network attachment.

* Remove redundant container detection messages.

* Limit DOCKER_HOST parsing to TCP.

* Restore docker socket existence check.

The check is skipped if the docker hostname is not localhost.

* Correct changelog entry..
(cherry picked from commit 3c2e8b99be)

Co-authored-by: Matt Clay <mclay@redhat.com>.
(cherry picked from commit 6362232c30)

Co-authored-by: Matt Clay <mclay@redhat.com>.
(cherry picked from commit 7c83f1f045)

Co-authored-by: Matt Clay <mclay@redhat.com>
5 years ago
Matt Clay d86d4285e1
CI stability updates. (#72307)
* Avoid issues with user-defined docker networks.
* Increase CI timeout.
* Increase unit test timeout.
5 years ago
Matt Clay 785fa40dc5 [stable-2.8] Fix ansible-test Azure Pipelines container auth.
(cherry picked from commit 2ef4b7e07e)

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
Rick Elrod b926f73309
[tests] --no-use-pep517 in kubevirt_conformance (#72016)
Change:
- This enables the inventory_kubevirt_conformance test to pass again on
  freebsd.
- This was due to a google-auth version bump. The dep chain looks like
  this: openshift -> kubernetes -> google-auth -> aiohttp -> multidict

Test Plan:
- ansible-test integration inventory_kubevirt_conformance --remote
  freebsd/12.0

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Matt Clay aac6ed13e6 [stable-2.8] Disabled inconsistent pylint checks..
(cherry picked from commit 8152d8bc1a)

Co-authored-by: Matt Clay <matt@mystile.com>
5 years ago
Matt Clay b8f6c2d4b9
[stable-2.8] CI provider fixes for ansible-test. (#71929) (#71934)
* Make Azure Pipelines resource_prefix lowercase.

* Make classification of CI files consistent.

* Update package-data sanity test for AZP..
(cherry picked from commit 92b66e3e31)

Co-authored-by: Matt Clay <mclay@redhat.com>
5 years ago
Rick Elrod 97c4064b87 [2.8] [tests] uninstall docker stuff for podman
Change:
- This isn't a direct backport of #71949 because in stable-2.8,
  setup_docker doesn't use handlers like more modern branches to clean
  up after itself.
- Instead, here we just make sure the docker packages are gone before
  the podman test runs.

Test Plan:
- CI
- ci_complete

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Rick Elrod 4616b3bbac
[incidental_setup_docker] changes to upstream repo (#71897) (#71901)
Change:
- The docker-ce.repo file for centos does not work on RHEL since it uses
  $releasever and on RHEL that is, e.g., "7Server".
- Instead, set up the repo manually.
- Additionally, the docker centos8 repo no longer has old versions, so
  we use the (only) version in the repo instead.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
(cherry picked from commit 31ddca4c0d)
(cherry picked from commit 651c0a2d03)
5 years ago
Matt Clay e35aafa81c [stable-2.8] Fix attribute testing in file integration test. (#71843)
The file test will no longer attempt to test attributes if `lsattr -vd` does not work on the system under test.
(cherry picked from commit 17765cd4e8)

Co-authored-by: Matt Clay <mclay@redhat.com>
5 years ago
Sam Doran caf82158d0 [stable-2.8] openssl_privatekey test - Add pause for macOS
The stat time granularity on macOS is one second. We recently upgraded
to faster macOS hosts, so some tests that run closely together to see if
something changed will have the same timestamp intermittently.
5 years ago
Sam Doran 81f660f606 [stable-2.8] incidental_setup_openssl - Pin version of Python packages
A recent update to cffi that was yanked is still being installed on our
Mac OS X 10.11 test image since the version of pip there is very old and
does not ignore yanked packages.

Pin the version of pyOpenSSL and its dependencies to fix this and avoid
future spontaneous failures..
(cherry picked from commit 65cdb86c8a)

Co-authored-by: Sam Doran <sdoran@redhat.com>
5 years ago
Matt Clay 909ac41247
[stable-2.8] Backport ansible-test CI provider support. (#71625)
* Add types.py from devel to support backport.

* [stable-2.9] Backport ansible-test CI provider support. (#71614)

* Add encoding.py from devel to support backports.

* Add io.py from devel to support backports.

* Update ansible-test support for CI providers. (#69522)

Refactored CI provider code to simplify multiple provider support and addition of new providers.

(cherry picked from commit d8e0aadc0d)

* Add Shippable request signing to ansible-test. (#69526)

(cherry picked from commit e7c2eb519b)

* ansible-test local change detection: use --base-branch if specified (#69508)

(cherry picked from commit 43acd61901)

* Add Azure Pipelines support to ansible-test.

(cherry picked from commit 8ffaed00f8)

* Update ansible-test remote endpoint handling. (#71413)

* Request ansible-core-ci resources by provider.
* Remove obsolete us-east-2 CI endpoint.
* Add new --remote-endpoint option.
* Add warning for --remote-aws-region option.
* Update service endpoints.
* Allow non-standard remote stages.
* Add changelog fragment.

(cherry picked from commit d099591964)

* Fix ansible-test coverage traceback. (#71446)

* Add integration test for ansible-test coverage.

* Fix ansible-test coverage traceback.

* Fix coverage reporting on Python 2.6.

(cherry picked from commit f5b6df14ab)

* Use new endpoint for Parallels based instances.

(cherry picked from commit 98febab975)

* Add pause to avoid same mtime in test.

(cherry picked from commit 3d769f3a76)

Co-authored-by: Felix Fontein <felix@fontein.de>

(cherry picked from commit 417e408f59)
5 years ago
Rick Elrod 83eda14330
[tests] fix fallout from dnf gpg fix (#71550)
Change:
- Missed needed gpg ignores

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Sam Doran a0ddefde38
Fix inventory_kubevirt_conformance test (#71545) 5 years ago
Rick Elrod 1fa2d5fd6b
[dnf] ensure packages are gpg-verified (#71541)
Change:
- By default the dnf API does not gpg-verify packages. This is a feature
  that is executed in its CLI code. It never made it into Ansible's
  usage of the API, so packages were previously not verified.
- This fixes CVE-2020-14365.

Test Plan:
- New integration tests

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Rick Elrod 555b1fdd7a
tests: Use `hg serve` instead of bitbucket for hg (#71403)
Change:
- Uses `hg serve` instead of a bitbucket repo for hg tests
- bitbucket no longer serves hg

Backport of #71398

Test Plan:
- CI, fixed integration tests

Signed-off-by: Rick Elrod <rick@elrod.me>
5 years ago
Matt Clay eddb0920e4 Revert "[stable-2.8] Cap pytest version to avoid relative import issue."
This reverts commit 9c5663699f.
6 years ago
Sam Doran c6eff6bdb2
Revert "[stable-2.8] Change default file permissions so they are not world readable (#70221) (#70827)" (#71233)
This reverts commit 11738aed97.
6 years ago
Rick Elrod eb1114a1b1
[tests] pin docker to <4.3.0 (#71193)
Change:
- New python docker lib wants a newer docker than we have in CI.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
6 years ago
David Shrewsbury f35e825440
[2.8] Sanitize URI module keys with no_log values (#70762) (#70822)
* Sanitize URI module keys with no_log values (#70762)

* Add sanitize_keys() to module_utils.

* More robust tests

* Revert 69653 change

* Allow list or dict

* fix pep8

* Sanitize lists within dict values

* words

* First pass at uri module

* Fix insane sanity tests

* fix integration tests

* Add changelog

* Remove unit test introduced in 69653

* Add ignore_keys param

* Sanitize all-the-things

* Ignore '_ansible*' keys

* cleanup

* Use module.no_log_values

* Avoid deep recursion issues by using deferred removal structure.

* Nit cleanups

* Add doc blurb

* spelling

* ci_complete

(cherry picked from commit bf98f031f3)

* update changelog for security_fixes

* Update 70762-sanitize-uri-keys.yml

Co-authored-by: Rick Elrod <rick@elrod.me>
6 years ago
Rick Elrod d73793fac7
Update integration tests to support rpmfluff-0.6 (#71155) (#71161)
Co-authored-by: Matt Martz <matt@sivel.net>
6 years ago
Sloane Hertel 291f94934c
copy - redact 'content' from invocation in check mode (#71033) (#71069)
* sanitize copy module invocation secrets in check mode

(cherry picked from commit 991714b9d1)
6 years ago
Sam Doran 11738aed97
[stable-2.8] Change default file permissions so they are not world readable (#70221) (#70827)
* [stable-2.8] Change default file permissions so they are not world readable (#70221)

* Change default file permissions so they are not world readable

CVE-2020-1736

Set the default permissions for files we create with atomic_move() to 0o0660. Track
which files we create that did not exist and warn if the module supports 'mode'
and it was not specified and the module did not call set_mode_if_different(). This allows the user to take action and specify a mode rather than using the defaults.

A code audit is needed to find all instances of modules that call atomic_move()
but do not call set_mode_if_different(). The findings need to be documented in
a changelog since we are not warning. Warning in those instances would be frustrating
to the user since they have no way to change the module code.

- use a set for storing list of created files
- just check the argument spac and params rather than using another property
- improve the warning message to include the default permissions.
(cherry picked from commit 5260527c4a)

Co-authored-by: Sam Doran <sdoran@redhat.com>

* Fix service test

* Fix lamdba_policy test

* Fix aws_lamdba test

* Fix warning for new default permissions when mode is not specified (#70976)

Follow up to #70221
Related to #67794
CVE-2020-1736

When set_mode_if_different() is called with mode of 'None', ensure we issue
a warning about the change in default permissions.

Add integration tests to ensure the warning works properly.

* Fix tests
- actually use custom module 🤦‍♂️
- verify file permission on created files
- use remote_tmp_dir so we're ready for split controller
- improve test module so we can skip the call to set_fs_attributes_if_different()
- fix tests for CentOS 6

(cherry picked from commit dc79528cc6)

* Use new category in changelog fragments
6 years ago
Sam Doran b95e520602
[stable-2.8] Drop "rhui-" prefix from RHEL repositories in AMI (#71130) (#71133)
The repository names seem to have changed and no longer have the "rhui-" prefix.
(cherry picked from commit 6ac4439a6a)

Co-authored-by: Sam Doran <sdoran@redhat.com>
6 years ago
Matt Clay 9c5663699f [stable-2.8] Cap pytest version to avoid relative import issue.
(cherry picked from commit 3a8ac62596)

Co-authored-by: Matt Clay <matt@mystile.com>
6 years ago
Sam Doran eb7597c5cb
[stable-2.8] Pin psutil version in pids test (#70716) 6 years ago
Sam Doran 9d7594c627
[stable-2.8] ansible_runner test - Add constraints (#70667) (#70671)
A recent updated to psutil, which is a dependency of ansible-runner, fails
to install on older versions of pip.

Commit with the breaking change:

  135628639b
(cherry picked from commit 9d27d7c8b1)

Co-authored-by: Sam Doran <sdoran@redhat.com>
6 years ago
Matt Clay 0f96e7e4cf [stable-2.8] Default ansible-test provisioning to us-east-1.
Previously windows instances on Shippable would be automatically directed to us-east-2.
(cherry picked from commit 1cf26896c5)

Co-authored-by: Matt Clay <matt@mystile.com>
6 years ago
Matt Clay 9fa105f183 Remove Azure module tests from CI. 6 years ago
Rick Elrod 0923e156c1 pip tests, use py2 compat sampleproject fork
Change:
- sampleproject has gone py3 only. Use a py2 compatible fork.

Test Plan:
- CI

Signed-off-by: Rick Elrod <rick@elrod.me>
6 years ago
Sam Doran b354b66c4b [stable-2.8] Pin version of netaddr and jmespath in filters test 6 years ago
Rick Elrod 14bcd3b7a3
yum - fix bug where enablerepo is not honored when disablerepo all (#66557) (#70085)
Fixes #66549

The inefficiency improvement
https://github.com/ansible/ansible/pull/63713 introduced a bug where
`enablerepo` was not being honored if combined with
`disablerepo="*"`. This fixes that issue.

Signed-off-by: Adam Miller <admiller@redhat.com>

Co-authored-by: Adam Miller <admiller@redhat.com>
6 years ago
Sam Doran 47256e921c
[stable-2.8] Pin Docker version at 19.03.1 (#69635) (#69646)
* [stable-2.8] Pin Docker version at 19.03.1
(cherry picked from commit fe941a4045)

Co-authored-by: Sam Doran <sdoran@redhat.com>

* [stable-2.8] Pin docker-ce-cli version in tests (#69620)

Installing docker-ce has a dependency of docker-ce-cli. If the version of docker-ci-cli is not specified, it installs the latest version.

(cherry picked from commit 889da811d7)
6 years ago
Matt Clay 4cdbcc4ed9 Disable failing azure_rm_dnszone test.
The module requires updates to work with the current Azure API.
6 years ago
Jordan Borean 85dfb1f49a Fix win_psmodule and win_psrepository tests 6 years ago
Rick Elrod e022b0d578 Add RHEL 8.2 to CI
Change:
RHEL 8.2 GA was released this week, test it in CI instead of 8.1.

Test Plan:
CI

Signed-off-by: Rick Elrod <rick@elrod.me>
6 years ago
Rick Elrod 94751ef971 Enable dnf integration tests on CentOS 8
Change:
We were only testing dnf on RHEL previously.
Test on CentOS 8 as well.

Test Plan:
Ran locally in docker.

Signed-off-by: Rick Elrod <rick@elrod.me>
6 years ago
Rick Elrod 75c5c543cb [dnf] Make behavior/errors compatible for new DNF
Change:
Extend the logic for custom error handling in the dnf module, so that on
newer DNF (such as DNF that ships with modern Fedora 31 container
images, and ships with RHEL 8.2) we report errors consistently with
older DNF.

Test Plan:
Ran dnf integration tests against an old Fedora 31 container image and a
brand new Fedora 32 container image; tess passed on both.

Signed-off-by: Rick Elrod <rick@elrod.me>
6 years ago
Rick Elrod d9d236ff56 [stable-2.8] pip - Fix check_mode for prerelease packages (#68690)
* pip - Fix check_mode for prerelease packages

Fixes #68592.

Signed-off-by: Rick Elrod <rick@elrod.me>
Co-authored-by: Matt Martz <matt@sivel.net>
(cherry picked from commit 82c60db49b)

Co-authored-by: Rick Elrod <rick@elrod.me>
6 years ago
Brian Coca 6f75aa2664 fixed fetch traversal from slurp (#68720)
* fixed fetch traversal from slurp

  * ignore slurp result for dest
  * fixed naming when source is relative
  * added tests with fake slurp
  * moved existing role tests into runme.sh
  * normalized on action excepts
  * moved dest transform down to when needed
  * added is_subpath check

fixes #67793

CVE-2019-3828

(cherry picked from commit ba87c225cd)
6 years ago
Sloane Hertel e6199d768c
[2.8] CVE-2020-1746 - Remove the params module option from ldap_attr and ldap_entry (#68715)
* Remove the params module option from ldap_attr and ldap_entry

Module options that circumvent Ansible's option handling were disallowed
in:
https://meetbot.fedoraproject.org/ansible-meeting/2017-09-28/ansible_dev_meeting.2017-09-28-15.00.log.html

Additionally, this particular usage can be insecure if bind_pw is set
this way as the password could end up in a logfile or displayed on
stdout.

Fixes CVE-2020-1746

(cherry picked from commit 0ff609f1bc)

* Fix formatting for option names

Co-Authored-By: Felix Fontein <felix@fontein.de>

* Fix fail_json

* update sanity

* fix indentation error

Co-authored-by: Toshio Kuratomi <a.badger@gmail.com>
Co-authored-by: Felix Fontein <felix@fontein.de>
6 years ago
Brian Coca 5eabf7bb93
prevent ansible_facts injection (#68431) (#68445)
* prevent ansible_facts injection (#68431)

- also only replace when needed
 - switched from replace to index
 - added test to verify bogus_facts are not accepted

CVE-2020-10684

(cherry picked from commit a9d2ceafe4)

* added to ignore
6 years ago
Brian Coca e1273b6faf fix vault temp file handling (#68433)
* fix vault tmpe file handling

 * use local temp dir instead of system temp
 * ensure each worker clears dataloader temp files
 * added test for dangling temp files
 * added notes to data loader

CVE-2020-10685

(cherry picked from commit 6452a82452)
6 years ago
Sloane Hertel 6c74a29870 subversion module - provide password securely when possible or warn (#67829)
* subversion module - provide password securely with svn command line option --password-from-stdin when possible, and provide a warning otherwise.
* Update lib/ansible/modules/source_control/subversion.py.
* Add a test.

Co-authored-by: Sam Doran <sdoran@redhat.com>
(cherry picked from commit d91658ec0c)
6 years ago
Roman 0e433f3d8f
[backport/2.8/63280] acl: fix module failure if there're spaces in a path (#63280) (#63809)
* acl: fix module failure if there're spaces in a path (#63280)

* Make acl module to work with whitespaces in path

* Added a changelog fragment

* Add quotes to changelog fragment

(cherry picked from commit 504d76e956)

* Fix tests
6 years ago
Sam Doran a46a9b8fac win_unzip - normalize and compare paths to prevent path traversal (#67799)
* Actually inspect the paths and prevent escape
* Add integration tests
* Generate zip files for use in integration test
* Adjust error message

(cherry picked from commit d30c57ab22)
6 years ago
Matt Clay 0a8d8b5c98 Disable failing azure_rm_cosmosdbaccount test. 6 years ago
Matt Clay 191a17a67f [stable-2.8] Update tests to use RHEL 7.8. (#68787)
* Update tests to use RHEL 7.8.

Keeping support for RHEL 7.6 since collections are still using it.

* Fix tests for RHEL 7.7+ due to extras repo name change..
(cherry picked from commit 04edd77c42)

Co-authored-by: Matt Clay <mclay@redhat.com>
6 years ago
Rick Elrod 0e2e863f5c Fix win_psrepository tests
Backport of a5414cb0e4

Signed-off-by: Rick Elrod <rick@elrod.me>
6 years ago
Rick Elrod 66cc931f0d
OpenSUSE15.1 container image + necessary test changes (2.8 edition) (#68794)
* add changelog fragment

Signed-off-by: Rick Elrod <rick@elrod.me>

* Update changelogs/fragments/ansible-test-opensuse-15.1.yml

Co-Authored-By: Matt Clay <matt@mystile.com>

* handle installing mysql on suse

Signed-off-by: Rick Elrod <rick@elrod.me>

* attempt to get tests passing again

Signed-off-by: Rick Elrod <rick@elrod.me>

* Update docker.txt to use the OpenSUSE 15.1 container image

Signed-off-by: Rick Elrod <rick@elrod.me>

Co-authored-by: Matt Clay <matt@mystile.com>
6 years ago