Dockerized test environment #747

jploskonka · 2017-10-13T17:06:09Z

I found it quite tricky to figure out how to run all codecept tests. I would really love to have it possible to be able to run all tests with one command. Also manually running php server for helpers tests was little bit irritating for me.

I use Docker a lot in development and don't want to install Selenium locally. @DavertMik I would like to get your opinion on that if it's something you'd consider merging and suggesting as simpler way to contribute.

If you like the idea I can cleanup the code and finish it sometime soon. There's still issue with Appium tests (I didn't have time to fit it yet).

Todo:

DavertMik · 2017-10-15T13:16:41Z

test/helper/Nightmare_test.js

 let should = require('chai').should();
 let I, browser;
-let site_url = 'http://127.0.0.1:8000';
+let site_url = 'http://php:8000';


(process.env.SITE_URL || 'http://127.0.0.1:8000');

here as well

DavertMik · 2017-10-15T13:21:03Z

Thanks @jploskonka . This is really great initiative. If you make it work with Appium I will adore you )
I couldn't make it run in Docker. It would be great to have headless mobile tests using Docker. If you make it work I'd add it to documentation as well...

Also, I'd propose to move docker-compose.yml into tests or rename it to docker-compose-test.yml so people knew this file is for internal testing, not for starting a project itself.

And there should be a flexible and documented way to run exact one helper and one test. As running all of them takes lot of time

jploskonka · 2017-10-16T20:26:01Z

I played a bit with Appium today and unfortunately wasn't able to run it :/ My container is always stuck on booting emulator so far:
Not sure if it's stuck or just takes so long to boot on my machine :D

I will give it one more shot this week, if I can't figure anything out I'll focus on getting rest of todos done ;) Still even without appium I think it's beneficial already and we can always work on Appium later. Thanks for your feedback!

Allow to pass test file path and run only it

DavertMik · 2017-10-17T21:45:16Z

I will give it one more shot this week, if I can't figure anything out I'll focus on getting rest of todos done ;) Still even without appium I think it's beneficial already and we can always work on Appium later. Thanks for your feedback!

Yep, and thanks for your input 🤗. That would be really great to have easy to setup test environment

docker-compose run test-helpers test/rest docker-compose run test-helpers test/rest/ApiDataFactory_test.js docker-compose run test-helpers test/rest/REST_test.js

Nightmare still failing on iframe examples

jploskonka · 2017-10-19T14:49:48Z

@DavertMik Another batch of updates, it's getting more interesting :D

I have problems with test/acceptance. Specs for WebDriver passes OK, but nightmare fails on iframe related examples. I couldn't find info about those specs in contributing.md and also it looks like those aren't run on travisCI nor semaphore. Am I doing something wrong or are those outdated and maybe broken?
I'm also not 100% convinced to separate docker-compose services for each acceptance test, but it's easier to debug it for now ;)

EDIT: Also I noticed that acceptance specs are tagged with @protractor while there's no protractor codecept config ;)

DavertMik · 2017-10-23T14:50:46Z

I have problems with test/acceptance. Specs for WebDriver passes OK, but nightmare fails on iframe related examples. I couldn't find info about those specs in contributing.md and also it looks like those aren't run on travisCI nor semaphore. Am I doing something wrong or are those outdated and maybe broken?

I didn't work with them for a long time so everything is possible 🙃
Please proceed, some tests may be disabled. Esp for Nightmare...

Also I noticed that acceptance specs are tagged with @protractor while there's no protractor codecept config ;)

I found it only here:

Scenario('within on form @WebDriverIO @protractor @nightmare', (I) => {

In this case I think @protractor corresponds to SeleniumWebdriver helper

…rized-codecept-tests

- Support passing NODE_VERSION as build arg - Reorder instructions to not reinstall npm packages if package.json doesn't change - Use COPY instead of ADD

- Run unit tests in docker container - Use generic language instead of node

- Use dockerized tests on travisCI - Share node_modules as volume between containers

…rized-codecept-tests

jploskonka · 2017-10-27T12:00:34Z

Almost done :D I still see issues mostly with travisCI integration but I'm not sure if this should be tackled in this PR or some other:

Builds against node 6.9.5 are failing on travis, I didn't experienced it locally yet. I'd guess those are timeouts issues but I'm not sure.
Because of building codecept container total build time on Travis is significantly longer
I no longer test no-docker version of codecept which may lead to some compatibility issues
Acceptance tests are not run on Travis, would be nice to tweak it (especially after WebDriverIO nested iframe support in the within block #755 got merged and added some useful tests there)
See builds https://travis-ci.org/Codeception/CodeceptJS/builds/293626424 and https://travis-ci.org/Codeception/CodeceptJS/builds/293621806

When I started working on this PR I didn't intended to use dockerized tests on travis yet, I did it only to check out how it would work and then couldn't resist going further :D I rolled back .travis.yml to version from master and builds are green. I think the best way would be to merge this and I'd like to work further on mentioned issues in separate PRs.

DavertMik · 2017-10-29T22:06:15Z

Thanks!
Maybe we could setup another CI for testing docker container only?
There are still few free CIs so why not to use them. I'd pick Wercker CI http://www.wercker.com as it's already based on Docker container and could be easily set up for it.

I created an account on Wercker and added you as admin. You should receive invitation.
You can use it to run docker tests. Not all CI tests but whatever you think is reasonable

jploskonka · 2017-11-01T12:36:23Z

I played a bit with Wercker and wasn't able to even start containers. I encountered some strange issues with privileges inside containers and gave up for now. I don't have much time this week to work on it ;)

Basically there's a lot of places we can improve this, but I'd like to add next things as separate PRs. Please let me know what do you think is must have to update here so we can get this merged and I can work on CI integration when I have some more time, probably next week :)

DavertMik · 2017-11-08T16:37:17Z

Thanks, let's switch to something new from Wercker

[dockerized tests/wip] Selenium, Protractor and Webdriver tests OK

12a979a

DavertMik reviewed Oct 15, 2017

View reviewed changes

jploskonka added 3 commits October 17, 2017 16:08

[docker-compose.yml] Don't run Appium tests

b75bfa6

[docker-compose.yml] Respect passed spec path

10b1efc

Allow to pass test file path and run only it

[TestHelper] Keep test configs in one place

dc82b4f

jploskonka force-pushed the dockerized-codecept-tests branch from 64fac63 to dc82b4f Compare October 17, 2017 19:25

[TestHelper] use default links friendly for Travis

140e48d

jploskonka force-pushed the dockerized-codecept-tests branch from 3367573 to 140e48d Compare October 17, 2017 20:22

jploskonka mentioned this pull request Oct 19, 2017

Can't run codecept docs locally. #752

Closed

jploskonka changed the title ~~[idea][work in progress] Dockerized development environment~~ [work in progress] Dockerized development environment Oct 19, 2017

[test/rest] Run in dockerized environment

43eb1c4

docker-compose run test-helpers test/rest docker-compose run test-helpers test/rest/ApiDataFactory_test.js docker-compose run test-helpers test/rest/REST_test.js

jploskonka force-pushed the dockerized-codecept-tests branch from f3e33d6 to 43eb1c4 Compare October 19, 2017 13:52

jploskonka added 2 commits October 19, 2017 16:12

dockerize WebDriverIO, Nightmare acceptance tests

a46010d

Nightmare still failing on iframe examples

[docker-compose] move env variables to .env file

9a2f9aa

Move docker-compose.yml and .env to /test dir

1ab6f30

jploskonka changed the title ~~[work in progress] Dockerized development environment~~ [work in progress] Dockerized test environment Oct 19, 2017

jploskonka added 3 commits October 25, 2017 13:46

Merge branch 'master' of github.com:Codeception/CodeceptJS into docke…

e877401

…rized-codecept-tests

[test/acceptance] Fix nightmare within tests

4c9f0be

[.travis.yml] Use dockerized helpers & rest tests

801a7d3

jploskonka force-pushed the dockerized-codecept-tests branch from ad62ba5 to 801a7d3 Compare October 25, 2017 13:22

jploskonka added 4 commits October 25, 2017 15:47

[.travis.yml] Remove php5-cli

7d1f5d0

[test/docker-compose] Add test-unit service

27694c5

[Dockerfile] Various fixes

313d758

- Support passing NODE_VERSION as build arg - Reorder instructions to not reinstall npm packages if package.json doesn't change - Use COPY instead of ADD

[.travis.yml] Fully dockerized tests

f57892f

- Run unit tests in docker container - Use generic language instead of node

jploskonka force-pushed the dockerized-codecept-tests branch from f8332e6 to d40680f Compare October 25, 2017 15:11

jploskonka force-pushed the dockerized-codecept-tests branch 5 times, most recently from fe05da3 to 6422d56 Compare October 25, 2017 16:39

jploskonka added 2 commits October 27, 2017 12:57

[Docker & docker-compose] Optimizations & travisCI

1f81f26

- Use dockerized tests on travisCI - Share node_modules as volume between containers

[test/acceptance] relative path to TestHelper

f7331fa

jploskonka force-pushed the dockerized-codecept-tests branch from 6422d56 to f7331fa Compare October 27, 2017 11:08

jploskonka added 2 commits October 27, 2017 13:10

Merge branch 'master' of github.com:Codeception/CodeceptJS into docke…

26221da

…rized-codecept-tests

[CONTRIBUTING.md] Add info about dockerized tests

4521a60

jploskonka changed the title ~~[work in progress] Dockerized test environment~~ Dockerized test environment Oct 27, 2017

[.travis.yml] check-out from master :D

8a89427

jploskonka force-pushed the dockerized-codecept-tests branch 3 times, most recently from ce89ccf to 393149e Compare November 1, 2017 10:40

wercker

3a04c01

jploskonka force-pushed the dockerized-codecept-tests branch from 393149e to 3a04c01 Compare November 1, 2017 10:43

DavertMik merged commit 65649ca into codeceptjs:master Nov 8, 2017

jploskonka deleted the dockerized-codecept-tests branch November 8, 2017 16:39

jploskonka mentioned this pull request Nov 9, 2017

Remove leftover wercker config files #771

Merged

jploskonka mentioned this pull request Nov 27, 2017

Circle ci: dockerized tests #796

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Dockerized test environment #747

Dockerized test environment #747

Uh oh!

jploskonka commented Oct 13, 2017 •

edited

Loading

Uh oh!

DavertMik Oct 15, 2017

Uh oh!

DavertMik commented Oct 15, 2017

Uh oh!

jploskonka commented Oct 16, 2017

Uh oh!

DavertMik commented Oct 17, 2017

Uh oh!

jploskonka commented Oct 19, 2017 •

edited

Loading

Uh oh!

DavertMik commented Oct 23, 2017

Uh oh!

jploskonka commented Oct 27, 2017

Uh oh!

DavertMik commented Oct 29, 2017 •

edited

Loading

Uh oh!

jploskonka commented Nov 1, 2017

Uh oh!

DavertMik commented Nov 8, 2017

Uh oh!

Uh oh!

Uh oh!

Dockerized test environment #747

Dockerized test environment #747

Uh oh!

Conversation

jploskonka commented Oct 13, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Todo:

Uh oh!

DavertMik Oct 15, 2017

Choose a reason for hiding this comment

Uh oh!

DavertMik commented Oct 15, 2017

Uh oh!

jploskonka commented Oct 16, 2017

Uh oh!

DavertMik commented Oct 17, 2017

Uh oh!

jploskonka commented Oct 19, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DavertMik commented Oct 23, 2017

Uh oh!

jploskonka commented Oct 27, 2017

Uh oh!

DavertMik commented Oct 29, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jploskonka commented Nov 1, 2017

Uh oh!

DavertMik commented Nov 8, 2017

Uh oh!

Uh oh!

jploskonka commented Oct 13, 2017 •

edited

Loading

jploskonka commented Oct 19, 2017 •

edited

Loading

DavertMik commented Oct 29, 2017 •

edited

Loading