Have builds only run when a certain branch is updated

Drone Support
#11

Does the drone.yml file have to exist in the master branch? Or can it exist in any branch?

We are evaluating drone ci and do not want to add additional files to master and would ideally keep this all contained within the branch labeled ‘test’.

#12

Yes the configuration file has to exist in the branch for the commit sha. If you want to override this behavior you can create a configuration extension, which can be used to customize how configuration files are fetched.

#13

So currently without bringing in custom extensions to the mix how are multiple environments each tied to their own branch on git handled?

e.g We have 3 environments
test -> dev -> production

Their github repos respectively are
test = test
dev = dev
production = master

It sounds to me like you guys suggest to store everything for every environment/branch within a single drone.yml that is within the master branch on git?

#14

If you push to branch master then Drone will fetch your yaml from master. If you push to branch develop then Drone will fetch your yaml from develop. I am happy to advise further but I’m not sure I have enough information to do so at this time.

#15

Nope no further explanation is needed, that behavior is sane and was what I was hoping for.

Thanks

#16

The trigger section really does not seem to work as intended.

In .drone.yml on branch test it has trigger section like so:

trigger:
  branch:
  - test

But on the drone server builds are triggering when anything is done on github remote even if the action has nothing to do with the branch ‘test’.
e.g People submit a pull request from branch foobar to master and a build starts up.

This behavior doesn’t make sense - If the trigger is set explicitly to branch: test why are all other actions not ignored entirely?

Example of a build triggering which has no relation to branch ‘test’

Number: 256
Status: success
Event: push
Commit: ab0d7302911055eea5695e9bcfccca0356923600
Branch: abzal_brazesubsaving
Ref: refs/heads/abzal_brazesubsaving
Author: mpostman
Message: IFNULL SubscriptionShippingSaving SUM set 0
#17

I cannot reproduce any issues with triggers. We are using this feature heavily in production without issue, but for good measure I ran tests to verify.

I pushed a commit to master and updated the configuration:

  commands:
  - hello
  - world

+trigger:
+  branch:
+  - develop

If you look at the build list you will see no build was triggered for this commit sha, demonstrating triggers are working as intended.

With that being said, I would encourage you to explore the underlying code if you feel strongly there is an issue with Drone. Here is a link to the relevant code and unit test:

#18

So this yml file is expected to work correctly if I only want builds to trigger from the branch ‘test’?

kind: pipeline
name: foobar
type: ssh

server:
  host:
    from_secret: test-env-vm_host
  user:
    from_secret: test-env-vm_user
  ssh_key:
    from_secret: test-env-vm_key

platform:
  os: linux
  arch: amd64

trigger:
  branch:
  - test

steps:
- name: deploy
  commands:
  - npm install
  - rsync --archive --delete --exclude=config.json . /home/user/foobar/
  - redis-cli flushdb
  - node --max-old-space-size=8192 /home/user/foobar/jobs/foobaz.js run
  - pm2 reload all
  environment:
    HOME: /home/user

Is there something I could be missing?

#19

Let’s assume you have this yaml file in branch master and in branch test:

kind: pipeline
name: default

steps:
- name: hello
  image: alpine
  commands:
  - echo hello world

trigger:
  branch:
  - test

and this yaml in branch develop with no triggers section:

kind: pipeline
name: default

steps:
- name: hello
  image: alpine
  commands:
  - echo hello world

Here are a few different scenarios that describe how the system will behave:

  • if you git push code to master
    • github sends a push hook
    • drone fetches the yaml from master
    • the target branch does not match branch test in the trigger section
    • no build is scheduled.
  • if you git push code to test
    • github sends a push hook
    • drone fetches the yaml from test
    • the target branch matches branch test in the trigger section
    • a new build is scheduled
  • if you git push code to develop
    • github sends a push hook
    • drone fetches the yaml from develop
    • there is no trigger section which means the push matches by default
    • a new build is scheduled.
  • if you open a pull request from test to master
    • github sends a pull request hook
    • drone fetches the yaml from the source branch test
    • the target branch is master and does not match branch test in the trigger section
    • no build is scheduled.
  • if you open a pull request from master to test
    • github sends a pull request hook
    • drone fetches the yaml from the target branch master
    • the target branch is test and matches branch test in the trigger section
    • a new build is scheduled.

So this yml file is expected to work correctly if I only want builds to trigger from the branch test?

I do not mean to be difficult but this question leaves some room for interpretation and it would be nice to clarify. Do you mean to say that you only want builds to execute when you git push to branch test? what does the yaml look like in branch test? what does the yaml look like in other branches?

#20

Thanks that description helps a lot.

It turns out people had created branches which contained a .drone.yml that was pushed to git without containing a trigger section to limit builds to the select branch.

Those examples you provided are very helpful though, if there are no such examples within the official docs it would be great to add them there.

#21

I can see there is a security issue with the way this works.

How would you stop a developer with malicious intent from pushing code to production?

If someone creates a branch and then modifies the .drone.yml they could put the same values for the SSH host/user/key for production servers, set the trigger to their own branch and then push their branch to github.

The drone server would then trigger a build against the values of the SSH host/user/key which would effectively push and override code on any given server.

How do you protect against this?

#22

If you do not trust your developers you should take the following steps:

  1. revoke permission to push directly to the repository
  2. require pull requests for all changes to be merged mainline
  3. conduct code reviews
  4. enable protected branches

This will prevent a malicious user from gaining access to a secret, since secrets are not exposed to pull requests by default. It will also prevent users from being able to modify the yaml without that being observed in a code review.

#23
  1. Restricting developers from being able to push their branches to github is unfeasible - many work from different machines and need to store their branches remotely.
  2. Changes to master are restricted and all pull requests undergo code review.
  3. See 2
  4. Not possible without hindering developer workflow.

So that said the only way to secure drone CI is to completely restrict access to master branch including read permissions and/or restrict branch pushing to github entirely.

#24

Restricting developers from being able to push their branches to github is unfeasible - many work from different machines and need to store their branches remotely.

I am not suggesting preventing them from pushing to a remote. I am suggesting using the fork workflow, where a developer forks the repository, pushes to their remote fork, and then opens a pull request to get their changes merged.

Alternatively you can extend secrets and put additional logic and restrictions in place. We have a starter project for creating extensions at https://github.com/drone/boilr-secret

changes to master are restricted

I did not realize it was possible to restrict pushing to specific branches, but I see this capability was added in Aug 2018. The way we designed secrets assumes any developer with push access can push to any branch, which was a true statement in 2014 when we created Drone, and was true for many years after.

However, seeing you can now restrict who can push to a branch, it makes sense to give the option to limit secrets by branch. This would need to be teed up as a new feature request.

#25

My suggestion would be to have the option to set the build target location and the build triggers at the server side and have the ability to restrict access to the drone server for essential persons only.

Regarding restricting access to the drone server one possible idea is look at repo/organization users and who has admin/owner rights then allow login/access to the drone server if a github user has admin/owner level permissions.

Should I open a feature request to discuss security around builds and their targets?

#26

I prefer to keep the pipeline configuration in the yaml. In this case secrets are already configured in the database, so I think it makes sense to add an option to limit a secret to a specific branch and align with github restricted branch feature.

If you want to move settings outside of the yaml for security purposes, the recommended approach is to create an extensions.

#27

Restricting secrets to certain branches would be a good fix here.

For our own use case here we only want master branch deployed to production SSH hosts so a simple server side option would restrict the use of certain secrets to specific branches would work.

But then how do you restrict access to the server to stop someone from changing that setting on the secret? Everyone with a github user at the organization the drone server is authed against can access and modify the server via the web portal.

#28

But then how do you restrict access to the server to stop someone from changing that setting on the secret?

I think it would be interesting to add a flag to lock and unlock the repository from editing. By default a repository would be unlocked. When locked, only admin users could edit.

I think this would solve the problem, however, I do not want to rush a design. Any features we add to core are subject to long term support, so I want to make sure anything we add is worth supporting for years to come.

#29

Understood.

For the time being we will have to remain with Jenkins until security is given more consideration for Drone CI.

It’s a shame too as Drone CI looks like a great platform especially its ‘out of box experience’.

I will check back in with this product sometime next year.

#30

You have options to use Drone today and lock down your environment without having to use a fork workflow as Brad suggested. Some options include:

  1. store secrets in vault which has the option to limit secrets by branch via drone/drone-vault#2
  2. create a validation extension (via drone/boilr-validate) to enforce rules specific to your organization including when and where secrets are used.

In the past there was no option to restrict pushing to branches in github, which made limiting secrets by branch ineffective security theater. It is great to learn that github supports restricted branches and we should definitely revise the way we do secrets to take advantage. None of our competitors (Travis, Circle, etc) seem to restrict secrets by branch at the moment, so this is a good opportunity to demonstrate that Drone continues to lead when it comes to security.