Unclear on GitHub activation

I have a multimachine deployment which appears to be set up appropriately. However I cannot activate any repos. I have read a good amount on the permissions and I want to know if the following error is stating the user does not have necessary permissions, and thus unable to create the necessary hooks.

I am also curious if GitHub scopes are something which needs to be modified for a deployment. My log error when attempting to activate is the following:

{"admin":true,"level":"debug","msg":"api: access granted","name":"some-service","namespace":"MyOrganization","read":true,"request-id":"123456789","time":"2019-02-13T18:56:48Z","user.login":"a-username","visibility":"public","write":true}

I get this as well the following errors in the browser console:

POST https://custom.myteam.ninja/api/repos/MyOrganization/some-service 500 Uncaught (in promise) SyntaxError: Unexpected token I in JSON at position 0

can you look at the networking tools in the browser and snapshot the http response you are receiving, and paste in this thread? also, I would expect more log data for activating a repository in the event of a server-side error … drone logs all error scenarios. Also, do you have a reverse proxy or load balancer in front of drone that could be intercepting or mutating traffic?

Also, what version of Drone are you using? Can you confirm you are using 1.0.0-rc.5?

Yes this is 1.0.0-rc.5

The entire response is:

Internal Server Error

Other requests seem fine and give a more typical response (for instance the request to the service page under apis/repos. Checking the docker logs I am also getting a invalid memory address. Possible this is after the first attempt to activate. I can post that stacktrace.

I do have an AWS alb in front of drone. In addition I am doing service discovery and container scheduling via Hashi Nomad/Consul.

yes, the more detail you can provide (logs, stacktrace, etc) would be helpful. A plain text response is abnormal, so I do not think this is related to permissions. But logs and stacktrace may help.

Following is the stacktrace of my error as well as the preceding 3 log messages:

{"fields.time":"2019-02-13T21:07:24Z","latency":14078,"level":"debug","method":"GET","msg":"","remote":"10.218.0.124:10592","request":"/v1/test/ping","request-id":"546579413","time":"2019-02-13T21:07:24Z"}
{"level":"debug","msg":"api: read access granted","name":"some-service","namespace":"MyOrg","request-id":"123456789","time":"2019-02-13T21:07:25Z","user.login":"a-user","visibility":"public"}
{"admin":true,"level":"debug","msg":"api: access granted","name":"some-service","namespace":"MyOrg","read":true,"request-id":"987654321","time":"2019-02-13T21:07:25Z","user.login":"a-user","visibility":"public","write":true}
Panic: runtime error: invalid memory address or nil pointer dereference
goroutine 8074 [running]:
runtime/debug.Stack(0x48, 0x0, 0x0)
    /usr/local/go/src/runtime/debug/stack.go:24 +0xa7
runtime/debug.PrintStack()
    /usr/local/go/src/runtime/debug/stack.go:16 +0x22
github.com/drone/drone/vendor/github.com/go-chi/chi/middleware.Recoverer.func1.1(0xc42013d900, 0x1a48c00, 0xc4206237a0)
    /go/src/github.com/drone/drone/vendor/github.com/go-chi/chi/middleware/recoverer.go:28 +0x1b6
panic(0x167c680, 0x25a9580)
    /usr/local/go/src/runtime/panic.go:502 +0x229
github.com/drone/drone/internal/scm/hook.deleteHook(0x1a49f80, 0xc4200bcfc0, 0xc4203531e0, 0xc420690280, 0x17, 0xc420ada1e0, 0x22, 0x2, 0xc420ada1e0)
    /go/src/github.com/drone/drone/internal/scm/hook/util.go:24 +0x44
github.com/drone/drone/internal/scm/hook.replaceHook(0x1a49f80, 0xc4200bcfc0, 0xc4203531e0, 0xc420690280, 0x17, 0xc4209a2780, 0x22, 0xc4200bcfc0)
    /go/src/github.com/drone/drone/internal/scm/hook/util.go:15 +0x74
github.com/drone/drone/internal/scm/hook.(*service).Create(0xc4200bcf90, 0x1a49f80, 0xc4200bcfc0, 0xc420814280, 0x18785df, 0x4)
    /go/src/github.com/drone/drone/internal/scm/hook/hook.go:63 +0x294
github.com/drone/drone/internal/api/repos.HandleEnable.func1(0x1a48c00, 0xc4206237a0, 0xc420868200)
    /go/src/github.com/drone/drone/internal/api/repos/enable.go:83 +0x33a
net/http.HandlerFunc.ServeHTTP(0xc4200b05a0, 0x1a48c00, 0xc4206237a0, 0xc420868200)
    /usr/local/go/src/net/http/server.go:1947 +0x44
github.com/drone/drone/internal/api/acl.CheckAccess.func1.1(0x1a48c00, 0xc4206237a0, 0xc420868000)
    /go/src/github.com/drone/drone/internal/api/acl/check.go:127 +0x62d
net/http.HandlerFunc.ServeHTTP(0xc42014ec60, 0x1a48c00, 0xc4206237a0, 0xc420868000)
    /usr/local/go/src/net/http/server.go:1947 +0x44
github.com/drone/drone/vendor/github.com/go-chi/chi.(*ChainHandler).ServeHTTP(0xc42008ab00, 0x1a48c00, 0xc4206237a0, 0xc420868000)
    /go/src/github.com/drone/drone/vendor/github.com/go-chi/chi/chain.go:31 +0x52
github.com/drone/drone/vendor/github.com/go-chi/chi.(*Mux).routeHTTP(0xc42008eae0, 0x1a48c00, 0xc4206237a0, 0xc420868000)
    /go/src/github.com/drone/drone/vendor/github.com/go-chi/chi/mux.go:424 +0x289
github.com/drone/drone/vendor/github.com/go-chi/chi.(*Mux).(github.com/drone/drone/vendor/github.com/go-chi/chi.routeHTTP)-fm(0x1a48c00, 0xc4206237a0, 0xc420868000)
    /go/src/github.com/drone/drone/vendor/github.com/go-chi/chi/mux.go:368 +0x48
net/http.HandlerFunc.ServeHTTP(0xc4203416f0, 0x1a48c00, 0xc4206237a0, 0xc420868000)
    /usr/local/go/src/net/http/server.go:1947 +0x44
github.com/drone/drone/internal/api/acl.CheckAccess.func1.1(0x1a48c00, 0xc4206237a0, 0xc420868000)
    /go/src/github.com/drone/drone/internal/api/acl/check.go:89 +0xb11
net/http.HandlerFunc.ServeHTTP(0xc42014e5d0, 0x1a48c00, 0xc4206237a0, 0xc420868000)
    /usr/local/go/src/net/http/server.go:1947 +0x44
github.com/drone/drone/internal/api/acl.InjectRepository.func1.1(0x1a48c00, 0xc4206237a0, 0xc42013df00)
    /go/src/github.com/drone/drone/internal/api/acl/repo.go:123 +0xa8b
net/http.HandlerFunc.ServeHTTP(0xc4200b0550, 0x1a48c00, 0xc4206237a0, 0xc42013df00)
    /usr/local/go/src/net/http/server.go:1947 +0x44
github.com/drone/drone/vendor/github.com/go-chi/chi.(*Mux).ServeHTTP(0xc42008eae0, 0x1a48c00, 0xc4206237a0, 0xc42013df00)
    /go/src/github.com/drone/drone/vendor/github.com/go-chi/chi/mux.go:69 +0x2eb
github.com/drone/drone/vendor/github.com/go-chi/chi.(*Mux).Mount.func1(0x1a48c00, 0xc4206237a0, 0xc42013df00)
    /go/src/github.com/drone/drone/vendor/github.com/go-chi/chi/mux.go:291 +0x107
net/http.HandlerFunc.ServeHTTP(0xc420116ca0, 0x1a48c00, 0xc4206237a0, 0xc42013df00)
    /usr/local/go/src/net/http/server.go:1947 +0x44
github.com/drone/drone/vendor/github.com/go-chi/chi.(*Mux).routeHTTP(0xc42008ea80, 0x1a48c00, 0xc4206237a0, 0xc42013df00)
    /go/src/github.com/drone/drone/vendor/github.com/go-chi/chi/mux.go:424 +0x289
github.com/drone/drone/vendor/github.com/go-chi/chi.(*Mux).(github.com/drone/drone/vendor/github.com/go-chi/chi.routeHTTP)-fm(0x1a48c00, 0xc4206237a0, 0xc42013df00)
    /go/src/github.com/drone/drone/vendor/github.com/go-chi/chi/mux.go:368 +0x48
net/http.HandlerFunc.ServeHTTP(0xc420341a30, 0x1a48c00, 0xc4206237a0, 0xc42013df00)
    /usr/local/go/src/net/http/server.go:1947 +0x44
github.com/drone/drone/vendor/github.com/go-chi/cors.(*Cors).Handler.func1(0x1a48c00, 0xc4206237a0, 0xc42013df00)
    /go/src/github.com/drone/drone/vendor/github.com/go-chi/cors/cors.go:199 +0xfe
net/http.HandlerFunc.ServeHTTP(0xc420116cc0, 0x1a48c00, 0xc4206237a0, 0xc42013df00)
    /usr/local/go/src/net/http/server.go:1947 +0x44
github.com/drone/drone/internal/api/auth.HandleAuthentication.func1.1(0x1a48c00, 0xc4206237a0, 0xc42013da00)
    /go/src/github.com/drone/drone/internal/api/auth/auth.go:41 +0x280
net/http.HandlerFunc.ServeHTTP(0xc4202ad350, 0x1a48c00, 0xc4206237a0, 0xc42013da00)
    /usr/local/go/src/net/http/server.go:1947 +0x44
github.com/drone/drone/internal/logger.Middleware.func1(0x1a48c00, 0xc4206237a0, 0xc42013d900)
    /go/src/github.com/drone/drone/internal/logger/handler.go:22 +0x220
net/http.HandlerFunc.ServeHTTP(0xc420116ce0, 0x1a48c00, 0xc4206237a0, 0xc42013d900)
    /usr/local/go/src/net/http/server.go:1947 +0x44
github.com/drone/drone/vendor/github.com/go-chi/chi/middleware.NoCache.func1(0x1a48c00, 0xc4206237a0, 0xc42013d900)
    /go/src/github.com/drone/drone/vendor/github.com/go-chi/chi/middleware/nocache.go:54 +0x217
net/http.HandlerFunc.ServeHTTP(0xc420116d00, 0x1a48c00, 0xc4206237a0, 0xc42013d900)
    /usr/local/go/src/net/http/server.go:1947 +0x44
github.com/drone/drone/vendor/github.com/go-chi/chi/middleware.Recoverer.func1(0x1a48c00, 0xc4206237a0, 0xc42013d900)
    /go/src/github.com/drone/drone/vendor/github.com/go-chi/chi/middleware/recoverer.go:35 +0x9f
net/http.HandlerFunc.ServeHTTP(0xc420116d20, 0x1a48c00, 0xc4206237a0, 0xc42013d900)
    /usr/local/go/src/net/http/server.go:1947 +0x44
github.com/drone/drone/vendor/github.com/go-chi/chi.(*Mux).ServeHTTP(0xc42008ea80, 0x1a48c00, 0xc4206237a0, 0xc42013d900)
    /go/src/github.com/drone/drone/vendor/github.com/go-chi/chi/mux.go:69 +0x2eb
github.com/drone/drone/vendor/github.com/go-chi/chi.(*Mux).Mount.func1(0x1a48c00, 0xc4206237a0, 0xc42013d900)
    /go/src/github.com/drone/drone/vendor/github.com/go-chi/chi/mux.go:291 +0x107
net/http.HandlerFunc.ServeHTTP(0xc420117ce0, 0x1a48c00, 0xc4206237a0, 0xc42013d900)
    /usr/local/go/src/net/http/server.go:1947 +0x44
github.com/drone/drone/vendor/github.com/go-chi/chi.(*Mux).routeHTTP(0xc42008e8a0, 0x1a48c00, 0xc4206237a0, 0xc42013d900)
    /go/src/github.com/drone/drone/vendor/github.com/go-chi/chi/mux.go:424 +0x289
github.com/drone/drone/vendor/github.com/go-chi/chi.(*Mux).(github.com/drone/drone/vendor/github.com/go-chi/chi.routeHTTP)-fm(0x1a48c00, 0xc4206237a0, 0xc42013d900)
    /go/src/github.com/drone/drone/vendor/github.com/go-chi/chi/mux.go:368 +0x48
net/http.HandlerFunc.ServeHTTP(0xc420341600, 0x1a48c00, 0xc4206237a0, 0xc42013d900)
    /usr/local/go/src/net/http/server.go:1947 +0x44
github.com/drone/drone/vendor/github.com/go-chi/chi.(*Mux).ServeHTTP(0xc42008e8a0, 0x1a48c00, 0xc4206237a0, 0xc42080f500)
    /go/src/github.com/drone/drone/vendor/github.com/go-chi/chi/mux.go:81 +0x221
net/http.serverHandler.ServeHTTP(0xc420353ee0, 0x1a48c00, 0xc4206237a0, 0xc42080f500)
    /usr/local/go/src/net/http/server.go:2694 +0xbc
net/http.(*conn).serve(0xc4205ecb40, 0x1a49ec0, 0xc4203f54c0)
    /usr/local/go/src/net/http/server.go:1830 +0x651
created by net/http.(*Server).Serve
    /usr/local/go/src/net/http/server.go:2795 +0x27b

Thanks, this was helpful. Can you share your Drone server configuration? Specifically your source code management configuration (e.g. GitHub) with the tokens redacted? These lines of code are the source of the stacktrace:

	u, _ := url.Parse(target)
	h, err := findHook(ctx, client, repo, u.Host)

drone assumes your source control management url is valid. This stacktrace tells me that parsing the url failed which caused a nil pointer, which is unexpected. My guess is that this is the root cause of all issues.

This is the config copied from our configs.

"DRONE_SERVER_HOST": "${PRIVATE_IP}",
"DRONE_LOGS_DEBUG": "true",
"DRONE_RPC_SECRET": "789456123",
"DRONE_GITHUB_PRIVATE_MODE": "true",
"DRONE_GITHUB_SERVER": "https://drone.my-organization.ninja",
"DRONE_GITHUB_CLIENT_ID": "123456789",
"DRONE_GITHUB_CLIENT_SECRET": "789456123",
"DRONE_DATABASE_DRIVER": "postgres",
"DRONE_DATABASE_DATASOURCE": "postgres://drone_b:[email protected]:5432/droneb"

Actually, I was wrong, the url.Parse that is failing is parsing your Drone server address. This is the combination of the DRONE_SERVER_HOST and DRONE_SERVER_PROTO. I would need to see these values to advise further.

Alternatively, if you do not want to share publicly you could try plugging them into the following script which could help surface any errors:
https://play.golang.org/p/UHJ4hWFjaJQ

If it is still unclear what the problem is, I would probably need to see the results of docker inspect <container> to see exactly what DRONE_SERVER_HOST and optionally DRONE_SERVER_PROTO are being set to.

Thanks for your help I will be trying those out tomorrow morning.

Checking in to say that the problem was from my environment. It was essentially a problem with Nomad env variables and the specifics of our base container image.

Thanks for your help as I never would have found the error without being pointed to the url.Parse error that stacktrace brought up.

awesome, glad to hear you figured it out