We face a weird issue recently, some pipelines missing
node_moudles that had been installed in previous steps.
I’ve tried to add a temp folder for each step but it doesn’t work.
by the way, after restarting the pipeline this step pass with no issue. thoughts?
/drone/src is a volume that persists between steps in the same pipeline which means the node_modules folder is available to subsequent steps . I created a simple test and was unable to reproduce the issue you are describing . I can confirm there have been no changes or regressions that would prevent this from working as expected.
I have attached a screenshot below that demonstrates my successful test. I have also included links below so that you can see our sample code and configuration, and compare with your own.
The only root cause I am aware of for the issue you described would be yaml misconfiguration. We are happy to help advise, but would kindly request a sample yaml and repository that we can use to reproduce the problem (similar to the sample repository and yaml we provided at the links below).
It’s working for 90% of the pipelines, but some of them missing this shared folder I guess. how can I verify this folder is shared between each step?
I am not aware of any instance in which the shared folder would be missing (unless you somehow added a yaml configuration that accidentally conflicted with the shared folder). The shared folder contains the source code cloned from your git repository. If the shared folder was missing, your source code would be missing too.
If the shared folder were missing, and your source code was missing, the npm install would fail due to missing package.json file. This does not seem to be the case based on the error message in your screenshot.
yes, that makes sense. I’ll dig into the yaml configuration. can I share it here just to double-check?
yes, you can share a copy of your yaml in this forum.
this is the problematic pipeline, please let me know if you spot any issue.
- name: Install dev dependencies
- npm ci
- name: Cancel running regressions
- node scripts/drone/cancel_running_regression.js
- Install dev dependencies
- name: Run lint
- npm run lint
- Cancel running regressions
- name: Run regression test
- echo "REGRESSION_STATUS=failure" > /tmp/.env
- npm run test
- rm -rf /tmp/.env
- echo "REGRESSION_STATUS=success" > /tmp/.env
TEST_DATABASE_URL: postgres://xxxx:[email protected]:5432/zzzz
- name: env
- Run lint
- name: database
docker-entrypoint.sh -c 'min_wal_size=2GB' -c 'min_wal_size=4GB' -c 'shared_buffers=4GB' -c 'effective_cache_size=4GB' -c 'wal_buffers=16MB' -c 'maintenance_work_mem=1GB' -c 'fsync=off' -c 'synchronous_commit=off' -c 'full_page_writes=off' -c 'max_connections=150' -c 'client_min_messages=warning'
- name: env
I’ve just noticed that some pipelines shared the same name, is that might be the issue?
any update here ? @bradrydzewski
@Amit_Baranes what happens if you change
npm ci to
In my head it is a fresh docker container and install each time the pipeline is run
based on the docs npm-ci | npm Docs - there needs to be a package-lock.json, but I don’t think there would be a package-lock.json because it is a fresh install
apart from that is there any way there could be a jest version mismatch - this feels like more of a node dependency mismatch issue than a drone issue
there is a package-lock.json file since we run
npm ci for all our pipelines. the thing is it doesn’t happen all the time. it might happen once \ twice a day or 15 times a day. also, it’s not always
jest module but others as well.
as for now, I have 2 guesses -
- we do something wrong on the pipeline level.
- nodes having issues passing this sharing folder between steps.
any update here? @bradrydzewski