Drone only shows 10 repositories from Gitea

ashwilliams1 · March 6, 2020, 7:20pm

hmm I cannot reproduce and we have not received any other reports of this issue (many Gitea maintainers also use Drone). I see 10 Gitea repositories on my dashboard and then I click the “Show all repositories” button at the bottom of the screen and it expands to show all of my repositories.

I can also access the user repository endpoint and see a list of all my repositories:
https://docs.drone.io/api/repos/repo_list/

ich777 · March 6, 2020, 8:05pm

Yes, i’ve tested this in the cloud version of drone.io with my github account and it reads all the repositories and i can click ‘Show all repositories’.
But if i install it on my server and also the docker runner it only loads 10 repos and the button ‘Show all repositories’ is not there.
I can only access the repositories that are loaded and nothing more.
The strange thing is that if i delete a few .drone.yml files so that i have less then 10 repos with the drone files and i click on sync it syncs the repos with the drone files in it and i can access them.
Can i post somewhere a verbose log? Also can someone tell my how to turn on the full log?

ashwilliams1 · March 6, 2020, 8:20pm

You can find logging options at docs.drone.io

ich777 · March 7, 2020, 12:31pm

Here is a screenshot from the get command:

Here is the link to the log:

Click

ich777 · March 7, 2020, 12:32pm

A screenshot from the main Drone page (there is no ‘Show all repositories’):

ich777 · March 7, 2020, 12:33pm

A screenshot from my Gitea repositories:

(Sorry had to do this in 3 posts, can only upload one picture per post)

ashwilliams1 · March 7, 2020, 6:58pm

that seems unexpected. I recommend enabling trace logging and looking at the Drone repository sync code to learn more [1]. As I am unable to reproduce any issues with Drone + Gitea there is not much we can do other than agree to review any patches that you submit, should you find a problem with Drone.

[1] https://github.com/drone/drone/tree/master/service/syncer

ich777 · March 7, 2020, 7:46pm

Sorry but i’m not much into coding with GO, so i will switch to something other than Drone.
Anyway thanks for the replys.

ich777 · March 8, 2020, 3:07pm

I’ve now tested it with Github and everything is working as expected.
I think that it has something to do with the Gitea plugin but since i’m not really into this i can’t help.

rafipiccolo · May 26, 2020, 2:20pm

Same bug for me : I have 3 docker containers on one host : gitea latest + drone latest + drone runner latest

It must be a mistake with pagination when calling gitea api.
I don’t know GO either so i dont know how i could help but can you confirm that the code is calling /user/repos
then /user/repos?page=2 and so on ?

If not i have a very easy workarround. You could set the limit to 50.
/user/repos?limit=50
could resolve my problem because i only have like 20 projects.
until the pagination mistake is solved.

please let me know if it is ok.

hokamoto · June 10, 2020, 12:53am

I just installed the latest Drone and Gitea today, and then got the same issue. Only 10 repos are shown up and there is no “Show all repositories”.

ich777 · June 10, 2020, 3:11pm

I have switched to Jenkins because no one is interested in solving the problem. Sadly I can’t help.

ashwilliams1 · June 10, 2020, 3:17pm

Just a reminder that Drone is open source and, if you are experiencing issues, you are encouraged to send a pull request.

bjj · June 10, 2020, 6:06pm

FYI on the gitea side it was probably caused by this change: https://github.com/go-gitea/gitea/pull/9452 and the impact on Drone is being discussed here: https://github.com/go-gitea/gitea/issues/11800

BTW it is possible to use drone+gitea in the current state because the gitea pagination returns most-recently-changed first, so you can work on recent repos.

ashwilliams1 · June 10, 2020, 6:15pm

Yes, looks like a regression of sorts since the integration was created before pagination existed. Adding pagination to the codebase should be relatively straight forward.

Pagination would need to be added to the code here:

github.com

drone/go-scm/blob/master/scm/driver/gitea/repo.go#L42


	return convertHook(out), res, err
}

func (s *repositoryService) FindPerms(ctx context.Context, repo string) (*scm.Perm, *scm.Response, error) {
	path := fmt.Sprintf("api/v1/repos/%s", repo)
	out := new(repository)
	res, err := s.client.do(ctx, "GET", path, nil, out)
	return convertRepository(out).Perm, res, err
}

func (s *repositoryService) List(ctx context.Context, _ scm.ListOptions) ([]*scm.Repository, *scm.Response, error) {
	path := fmt.Sprintf("api/v1/user/repos")
	out := []*repository{}
	res, err := s.client.do(ctx, "GET", path, nil, &out)
	return convertRepositoryList(out), res, err
}

func (s *repositoryService) ListHooks(ctx context.Context, repo string, _ scm.ListOptions) ([]*scm.Hook, *scm.Response, error) {
	path := fmt.Sprintf("api/v1/repos/%s/hooks", repo)
	out := []*hook{}
	res, err := s.client.do(ctx, "GET", path, nil, &out)

For reference you can see how we do this with GitHub:
https://github.com/drone/go-scm/blob/master/scm/river/github/repo.go#L85

Some additional code may be required to extract the pagination data from the response (i.e. current page, next page, last page, results per page). However, if Gitea uses the same approach as GitHub (example below) than this would not require any additional code.

Link: <https://api.github.com/user/repos?page=3&per_page=100>; rel="next",
  <https://api.github.com/user/repos?page=50&per_page=100>; rel="last"

bjj · June 10, 2020, 8:45pm

The gitea folks seem to be working on this from multiple angles, but as of right now I pulled gitea/gitea:latest (self-identifies as 1.13.0+dev) and drone can see all of my repos without any reconfiguration. I didn’t dig into the changelists, so this may only be allowing configuration of the default page size (making it larger than 10) but it should be enough to get people going.

bradrydzewski · June 10, 2020, 9:10pm

@bjj thanks for the details. I created the following issue to track:

ryanjaeb · June 13, 2020, 7:53pm

I posted this in the issue, but it doesn’t look like anyone is subscribed to that. I hope it’s ok for me to cross post here.

There’s a Gitea PR that says it fixed this to work without any Drone changes. I don’t have 10+ repos to test it easily, but, for anyone that was having a problem, the linux-amd64 Docker image looks like it already includes the changes (org.label-schema.vcs-ref=48842ed1).

moepman · June 18, 2020, 9:56pm

I have setup a gitea instance using the just released Gitea 1.12.0, added drone via oauth, logged in with my account that is having >30 repos … and I found the syncing icon to just spin forever.

I can’t find anything useful in the drone log (DRONE_LOGS_DEBUG set to true already).

Gitea however displays multiple requests to /api/v1/user/repos per second for minutes (until I just stop drone).

Please let me know how I can help debuging this issue.

moepman · June 23, 2020, 8:42am

Could someone please bump the go-scm dependency in drone to include this change: https://github.com/drone/go-scm/pull/66?
That should fix problems related to pagination and gitea.