Commit graph

189 commits

Author SHA1 Message Date
Loïc Dachary
a8be33da60
[CI] tests/pgsql.ini.tmpl: do not use minio
f92e0a4018 added minio to the tests/pgsql.ini.tmpl and Forgejo CI does
not run a minio server. It will if there are external storage related
changes at some point but it is not the case now so it is not worth
the burden.

(cherry picked from commit 4cfbf4718d)
(cherry picked from commit 9bd644d601)
(cherry picked from commit 604636c7c4)
(cherry picked from commit 341cdb8540)
(cherry picked from commit 57bdc91de5)
(cherry picked from commit c6ec6517b4)
(cherry picked from commit 2f03fc1b29)
(cherry picked from commit aaecd10966)
(cherry picked from commit 7025ff7b06)
(cherry picked from commit 15e6485ae1)
(cherry picked from commit 5f7b23c659)
(cherry picked from commit a74d7ce6c5)
(cherry picked from commit f552e2fc56)
(cherry picked from commit 9fa4a90baa)
2023-06-23 09:27:42 +02:00
Loïc Dachary
b1c86fb777
[CI] set PASSWORD_HASH_ALGO = argon2 for integration tests
(cherry picked from commit 1d7ce2a39c)
(cherry picked from commit 3af8757583)
(cherry picked from commit 0b81815209)
(cherry picked from commit 7180a3222a)
(cherry picked from commit 069044f4a6)
(cherry picked from commit dd1438f389)
(cherry picked from commit 4ae6b5d64d)
(cherry picked from commit c572666d78)
(cherry picked from commit de0a4480bc)
(cherry picked from commit 60272ef4f7)
(cherry picked from commit a68fdde4be)
(cherry picked from commit ff448cf3a6)
(cherry picked from commit 208498caaa)
(cherry picked from commit 1abd3efad8)
(cherry picked from commit 818c7f4ca9)
(cherry picked from commit 1e552bbe44)
(cherry picked from commit 88dbd6da1d)
2023-06-23 09:27:42 +02:00
silverwind
d2142ba3c3
Update octicons and use octicon-file-directory-symlink (#25453)
Make use of the [new
octicon](https://github.com/primer/octicons/issues/945) that indicates a
symlink to a directory:

<img width="189" alt="Screenshot 2023-06-22 at 22 50 57"
src="https://github.com/go-gitea/gitea/assets/115237/a70690ea-ebfc-48fe-af23-cdc33bcb2098">
2023-06-22 22:05:52 +00:00
wxiaoguang
2cdf260f42
Refactor path & config system (#25330)
# The problem

There were many "path tricks":

* By default, Gitea uses its program directory as its work path
* Gitea tries to use the "work path" to guess its "custom path" and
"custom conf (app.ini)"
* Users might want to use other directories as work path
* The non-default work path should be passed to Gitea by GITEA_WORK_DIR
or "--work-path"
* But some Gitea processes are started without these values
    * The "serv" process started by OpenSSH server
    * The CLI sub-commands started by site admin
* The paths are guessed by SetCustomPathAndConf again and again
* The default values of "work path / custom path / custom conf" can be
changed when compiling

# The solution

* Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use
test code to cover its behaviors.
* When Gitea's web server runs, write the WORK_PATH to "app.ini", this
value must be the most correct one, because if this value is not right,
users would find that the web UI doesn't work and then they should be
able to fix it.
* Then all other sub-commands can use the WORK_PATH in app.ini to
initialize their paths.
* By the way, when Gitea starts for git protocol, it shouldn't output
any log, otherwise the git protocol gets broken and client blocks
forever.

The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path
> env var GITEA_WORK_DIR > builtin default

The "app.ini" searching order is: cmd arg --config > cmd arg "work path
/ custom path" > env var "work path / custom path" > builtin default


## ⚠️ BREAKING

If your instance's "work path / custom path / custom conf" doesn't meet
the requirements (eg: work path must be absolute), Gitea will report a
fatal error and exit. You need to set these values according to the
error log.



----

Close #24818
Close #24222
Close #21606
Close #21498
Close #25107
Close #24981
Maybe close #24503

Replace #23301
Replace #22754

And maybe more
2023-06-21 13:50:26 +08:00
Kyle D
8220e50b56
Substitute variables in path names of template repos too (#25294)
### Summary

Extend the template variable substitution to replace file paths. This
can be helpful for setting up log files & directories that should match
the repository name.

### PR Changes

 - Move files matching glob pattern when setting up repos from template
- For security, added ~escaping~ sanitization for cross-platform support
and to prevent directory traversal (thanks @silverwind for the
reference)
 - Added unit testing for escaping function 
- Fixed the integration tests for repo template generation by passing
the repo_template_id
- Updated the integration testfiles to add some variable substitution &
assert the outputs

I had to fix the existing repo template integration test and extend it
to add a check for variable substitutions.

Example:

![image](https://github.com/go-gitea/gitea/assets/12700993/621feb09-0ef3-460e-afa8-da74cd84fa4e)
2023-06-20 21:14:47 +00:00
Zettat123
33cd74ad70
Fix LDAP sync when Username Attribute is empty (#25278)
Fix #21072

![image](https://github.com/go-gitea/gitea/assets/15528715/96b30beb-7f88-4a60-baae-2e5ad8049555)

Username Attribute is not a required item when creating an
authentication source. If Username Attribute is empty, the username
value of LDAP user cannot be read, so all users from LDAP will be marked
as inactive by mistake when synchronizing external users.

This PR improves the sync logic, if username is empty, the email address
will be used to find user.
2023-06-20 11:04:13 +08:00
wxiaoguang
b4e4b7ad51
Make backend code respond correct JSON when creating PR (#25353)
Fix #25351
2023-06-19 08:25:36 +00:00
wxiaoguang
4e2f1ee58d
Refactor web package and context package (#25298)
1. The "web" package shouldn't depends on "modules/context" package,
instead, let each "web context" register themselves to the "web"
package.
2. The old Init/Free doesn't make sense, so simplify it
* The ctx in "Init(ctx)" is never used, and shouldn't be used that way
* The "Free" is never called and shouldn't be called because the SSPI
instance is shared

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-06-18 09:59:09 +02:00
wxiaoguang
b71cb7acdc
Use fetch to send requests to create issues/comments (#25258)
Follow #23290

Network error won't make content lost. And this is a much better
approach than "loading-button".

The UI is not perfect and there are still some TODOs, they can be done
in following PRs, not a must in this PR's scope.

<details>


![image](https://github.com/go-gitea/gitea/assets/2114189/c94ba958-aa46-4747-8ddf-6584deeed25c)

</details>
2023-06-16 06:32:43 +00:00
Lunny Xiao
d6dd6d641b
Fix all possible setting error related storages and added some tests (#23911)
Follow up #22405

Fix #20703 

This PR rewrites storage configuration read sequences with some breaks
and tests. It becomes more strict than before and also fixed some
inherit problems.

- Move storage's MinioConfig struct into setting, so after the
configuration loading, the values will be stored into the struct but not
still on some section.
- All storages configurations should be stored on one section,
configuration items cannot be overrided by multiple sections. The
prioioty of configuration is `[attachment]` > `[storage.attachments]` |
`[storage.customized]` > `[storage]` > `default`
- For extra override configuration items, currently are `SERVE_DIRECT`,
`MINIO_BASE_PATH`, `MINIO_BUCKET`, which could be configured in another
section. The prioioty of the override configuration is `[attachment]` >
`[storage.attachments]` > `default`.
- Add more tests for storages configurations.
- Update the storage documentations.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-06-14 11:42:38 +08:00
wxiaoguang
6bbccdd177
Improve AJAX link and modal confirm dialog (#25210)
Clarify the "link-action" behavior:

>  // A "link-action" can post AJAX request to its "data-url"
> // Then the browser is redirect to: the "redirect" in response, or
"data-redirect" attribute, or current URL by reloading.

And enhance the "link-action" to support showing a modal dialog for
confirm. A similar general approach could also help PRs like
https://github.com/go-gitea/gitea/pull/22344#discussion_r1062883436

> // If the "link-action" has "data-modal-confirm(-html)" attribute, a
confirm modal dialog will be shown before taking action.


And a lot of duplicate code can be removed now. A good framework design
can help to avoid code copying&pasting.

---------

Co-authored-by: silverwind <me@silverwind.io>
2023-06-13 12:10:10 +00:00
yp05327
22a39bb961
Fix profile render when the README.md size is larger than 1024 bytes (#25131)
Fixes https://github.com/go-gitea/gitea/issues/25094

`GetBlobContent` will only get the first 1024 bytes, if the README.md
size is larger than 1024 bytes,
We can not render the rest of them.
After this fix, we should provide the limited size to read when call
`GetBlobContent`.

After:

![image](https://github.com/go-gitea/gitea/assets/18380374/22a42936-4cf8-40b4-a5c7-e384082beb0d)

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-06-13 09:02:25 +00:00
Lunny Xiao
315124b469
Fix parallelly generating index failure with Mysql (#24567) 2023-06-05 10:33:47 +00:00
Jack Hay
18de83b2a3
Redesign Scoped Access Tokens (#24767)
## Changes
- Adds the following high level access scopes, each with `read` and
`write` levels:
    - `activitypub`
    - `admin` (hidden if user is not a site admin)
    - `misc`
    - `notification`
    - `organization`
    - `package`
    - `issue`
    - `repository`
    - `user`
- Adds new middleware function `tokenRequiresScopes()` in addition to
`reqToken()`
  -  `tokenRequiresScopes()` is used for each high-level api section
- _if_ a scoped token is present, checks that the required scope is
included based on the section and HTTP method
  - `reqToken()` is used for individual routes
- checks that required authentication is present (but does not check
scope levels as this will already have been handled by
`tokenRequiresScopes()`
- Adds migration to convert old scoped access tokens to the new set of
scopes
- Updates the user interface for scope selection

### User interface example
<img width="903" alt="Screen Shot 2023-05-31 at 1 56 55 PM"
src="https://github.com/go-gitea/gitea/assets/23248839/654766ec-2143-4f59-9037-3b51600e32f3">
<img width="917" alt="Screen Shot 2023-05-31 at 1 56 43 PM"
src="https://github.com/go-gitea/gitea/assets/23248839/1ad64081-012c-4a73-b393-66b30352654c">

## tokenRequiresScopes  Design Decision
- `tokenRequiresScopes()` was added to more reliably cover api routes.
For an incoming request, this function uses the given scope category
(say `AccessTokenScopeCategoryOrganization`) and the HTTP method (say
`DELETE`) and verifies that any scoped tokens in use include
`delete:organization`.
- `reqToken()` is used to enforce auth for individual routes that
require it. If a scoped token is not present for a request,
`tokenRequiresScopes()` will not return an error

## TODO
- [x] Alphabetize scope categories
- [x] Change 'public repos only' to a radio button (private vs public).
Also expand this to organizations
- [X] Disable token creation if no scopes selected. Alternatively, show
warning
- [x] `reqToken()` is missing from many `POST/DELETE` routes in the api.
`tokenRequiresScopes()` only checks that a given token has the correct
scope, `reqToken()` must be used to check that a token (or some other
auth) is present.
   -  _This should be addressed in this PR_
- [x] The migration should be reviewed very carefully in order to
minimize access changes to existing user tokens.
   - _This should be addressed in this PR_
- [x] Link to api to swagger documentation, clarify what
read/write/delete levels correspond to
- [x] Review cases where more than one scope is needed as this directly
deviates from the api definition.
   - _This should be addressed in this PR_
   - For example: 
   ```go
	m.Group("/users/{username}/orgs", func() {
		m.Get("", reqToken(), org.ListUserOrgs)
		m.Get("/{org}/permissions", reqToken(), org.GetUserOrgsPermissions)
}, tokenRequiresScopes(auth_model.AccessTokenScopeCategoryUser,
auth_model.AccessTokenScopeCategoryOrganization),
context_service.UserAssignmentAPI())
   ```

## Future improvements
- [ ] Add required scopes to swagger documentation
- [ ] Redesign `reqToken()` to be opt-out rather than opt-in
- [ ] Subdivide scopes like `repository`
- [ ] Once a token is created, if it has no scopes, we should display
text instead of an empty bullet point
- [ ] If the 'public repos only' option is selected, should read
categories be selected by default

Closes #24501
Closes #24799

Co-authored-by: Jonathan Tran <jon@allspice.io>
Co-authored-by: Kyle D <kdumontnu@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
2023-06-04 20:57:16 +02:00
Denys Konovalov
7d855efb1f
Allow for PKCE flow without client secret + add docs (#25033)
The PKCE flow according to [RFC
7636](https://datatracker.ietf.org/doc/html/rfc7636) allows for secure
authorization without the requirement to provide a client secret for the
OAuth app.

It is implemented in Gitea since #5378 (v1.8.0), however without being
able to omit client secret.
Since #21316 Gitea supports setting client type at OAuth app
registration.

As public clients are already forced to use PKCE since #21316, in this
PR the client secret check is being skipped if a public client is
detected. As Gitea seems to implement PKCE authorization correctly
according to the spec, this would allow for PKCE flow without providing
a client secret.

Also add some docs for it, please check language as I'm not a native
English speaker.

Closes #17107
Closes #25047
2023-06-03 05:59:28 +02:00
Lunny Xiao
5d23c885ed
Fix users cannot visit issue attachment bug (#25019)
Caused by #24362

Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-31 19:06:17 +02:00
wxiaoguang
a90988d63f
Update repo's default branch when adding new files in an empty one (#25017)
Fix #25014

Only API needs this fix. On the Web UI, users could only add new file on
the default branch.
2023-05-31 17:07:51 +08:00
Yevhen Pavlov
970e1c98ec
Add v3.18 to TestPackageAlpine (#24972)
Add Alpine 3.18 to TestPackageAlpine

Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-29 15:45:32 +00:00
wxiaoguang
ca5f302876
Fix admin config page error, use tests to cover the admin config and 500 error page (#24965)
The admin config page has been broken for many many times, a little
refactoring would make this page panic.

So, add a test for it, and add another test to cover the 500 error page.

Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-29 15:00:21 +00:00
Denys Konovalov
275d4b7e3f
API endpoint for changing/creating/deleting multiple files (#24887)
This PR creates an API endpoint for creating/updating/deleting multiple
files in one API call similar to the solution provided by
[GitLab](https://docs.gitlab.com/ee/api/commits.html#create-a-commit-with-multiple-files-and-actions).

To archive this, the CreateOrUpdateRepoFile and DeleteRepoFIle functions
in files service are unified into one function supporting multiple files
and actions.

Resolves #14619
2023-05-29 17:41:35 +08:00
JakobDev
aaa1094663
Add the ability to pin Issues (#24406)
This adds the ability to pin important Issues and Pull Requests. You can
also move pinned Issues around to change their Position. Resolves #2175.

## Screenshots

![grafik](https://user-images.githubusercontent.com/15185051/235123207-0aa39869-bb48-45c3-abe2-ba1e836046ec.png)

![grafik](https://user-images.githubusercontent.com/15185051/235123297-152a16ea-a857-451d-9a42-61f2cd54dd75.png)

![grafik](https://user-images.githubusercontent.com/15185051/235640782-cbfe25ec-6254-479a-a3de-133e585d7a2d.png)

The Design was mostly copied from the Projects Board.

## Implementation
This uses a new `pin_order` Column in the `issue` table. If the value is
set to 0, the Issue is not pinned. If it's set to a bigger value, the
value is the Position. 1 means it's the first pinned Issue, 2 means it's
the second one etc. This is dived into Issues and Pull requests for each
Repo.

## TODO
- [x] You can currently pin as many Issues as you want. Maybe we should
add a Limit, which is configurable. GitHub uses 3, but I prefer 6, as
this is better for bigger Projects, but I'm open for suggestions.
- [x] Pin and Unpin events need to be added to the Issue history.
- [x] Tests
- [x] Migration

**The feature itself is currently fully working, so tester who may find
weird edge cases are very welcome!**

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-25 15:17:19 +02:00
JakobDev
25dc1556cd
Add API for Label templates (#24602)
This adds API that allows getting the Label templates of the Gitea
Instance
2023-05-23 18:10:23 +08:00
KN4CK3R
cdb088cec2
Add CRAN package registry (#22331)
This PR adds a [CRAN](https://cran.r-project.org/) package registry.

![grafik](https://user-images.githubusercontent.com/1666336/210450039-d6fa6f77-20cd-4741-89a8-1624def267f7.png)
2023-05-22 10:57:49 +08:00
wxiaoguang
4647660776
Rewrite logger system (#24726)
## ⚠️ Breaking

The `log.<mode>.<logger>` style config has been dropped. If you used it,
please check the new config manual & app.example.ini to make your
instance output logs as expected.

Although many legacy options still work, it's encouraged to upgrade to
the new options.

The SMTP logger is deleted because SMTP is not suitable to collect logs.

If you have manually configured Gitea log options, please confirm the
logger system works as expected after upgrading.

## Description

Close #12082 and maybe more log-related issues, resolve some related
FIXMEs in old code (which seems unfixable before)

Just like rewriting queue #24505 : make code maintainable, clear legacy
bugs, and add the ability to support more writers (eg: JSON, structured
log)

There is a new document (with examples): `logging-config.en-us.md`

This PR is safer than the queue rewriting, because it's just for
logging, it won't break other logic.

## The old problems

The logging system is quite old and difficult to maintain:
* Unclear concepts: Logger, NamedLogger, MultiChannelledLogger,
SubLogger, EventLogger, WriterLogger etc
* Some code is diffuclt to konw whether it is right:
`log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs
`log.DelLogger("console")`
* The old system heavily depends on ini config system, it's difficult to
create new logger for different purpose, and it's very fragile.
* The "color" trick is difficult to use and read, many colors are
unnecessary, and in the future structured log could help
* It's difficult to add other log formats, eg: JSON format
* The log outputer doesn't have full control of its goroutine, it's
difficult to make outputer have advanced behaviors
* The logs could be lost in some cases: eg: no Fatal error when using
CLI.
* Config options are passed by JSON, which is quite fragile.
* INI package makes the KEY in `[log]` section visible in `[log.sub1]`
and `[log.sub1.subA]`, this behavior is quite fragile and would cause
more unclear problems, and there is no strong requirement to support
`log.<mode>.<logger>` syntax.


## The new design

See `logger.go` for documents.


## Screenshot

<details>


![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff)


![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9)


![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee)

</details>

## TODO

* [x] add some new tests
* [x] fix some tests
* [x] test some sub-commands (manually ....)

---------

Co-authored-by: Jason Song <i@wolfogre.com>
Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-21 22:35:11 +00:00
Lunny Xiao
c59a057297
Refactor rename user and rename organization (#24052)
This PR is a refactor at the beginning. And now it did 4 things.
- [x] Move renaming organizaiton and user logics into services layer and
merged as one function
- [x] Support rename a user capitalization only. For example, rename the
user from `Lunny` to `lunny`. We just need to change one table `user`
and others should not be touched.
- [x] Before this PR, some renaming were missed like `agit`
- [x] Fix bug the API reutrned from `http.StatusNoContent` to `http.StatusOK`
2023-05-21 23:13:47 +08:00
silverwind
32d9c47ec7
Add RTL rendering support to Markdown (#24816)
Support RTL content in Markdown:


![image](https://github.com/go-gitea/gitea/assets/115237/dedb1b0c-2f05-40dc-931a-0d9dc81f7c97)

Example document:
https://try.gitea.io/silverwind/symlink-test/src/branch/master/bidi-text.md
Same on GitHub:
https://github.com/silverwind/symlink-test/blob/master/bidi-text.md

`dir=auto` enables a browser heuristic that sets the text direction
automatically. It is the only way to get automatic text direction.

Ref: https://codeberg.org/Codeberg/Community/issues/1021

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-05-20 23:02:52 +02:00
FuXiaoHei
c757765a9e
Implement actions artifacts (#22738)
Implement action artifacts server api.

This change is used for supporting
https://github.com/actions/upload-artifact and
https://github.com/actions/download-artifact in gitea actions. It can
run sample workflow from doc
https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts.
The api design is inspired by
https://github.com/nektos/act/blob/master/pkg/artifacts/server.go and
includes some changes from gitea internal structs and methods.

Actions artifacts contains two parts:

- Gitea server api and storage (this pr implement basic design without
some complex cases supports)
- Runner communicate with gitea server api (in comming)

Old pr https://github.com/go-gitea/gitea/pull/22345 is outdated after
actions merged. I create new pr from main branch.


![897f7694-3e0f-4f7c-bb4b-9936624ead45](https://user-images.githubusercontent.com/2142787/219382371-eb3cf810-e4e0-456b-a8ff-aecc2b1a1032.jpeg)

Add artifacts list in actions workflow page.
2023-05-19 21:37:57 +08:00
a1012112796
25d4f95df2
replace drone exec to act_runner exec in test README.md (#24791) 2023-05-18 19:48:47 +00:00
silverwind
e720f49206
Skip TestRepoCommitsStatusParallel on CI (#24741)
Related: https://github.com/go-gitea/gitea/issues/22109

Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-16 14:42:16 +02:00
KN4CK3R
5968c63a11
Add Go package registry (#24687)
Fixes #7608

This PR adds a Go package registry usable with the Go proxy protocol.

![grafik](https://github.com/go-gitea/gitea/assets/1666336/328feb5c-3df2-4f9d-8eae-fe3126d14c37)
2023-05-14 23:38:40 +08:00
Lunny Xiao
68081c4721
Add test for api team orgnization (#24699)
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-13 21:26:35 +00:00
KN4CK3R
9173e079ae
Add Alpine package registry (#23714)
This PR adds an Alpine package registry. You can follow [this
tutorial](https://wiki.alpinelinux.org/wiki/Creating_an_Alpine_package)
to build a *.apk package for testing.

This functionality is similar to the Debian registry (#22854) and
therefore shares some methods. I marked this PR as blocked because it
should be merged after #22854.


![grafik](https://user-images.githubusercontent.com/1666336/227779595-b76163aa-eea1-4a79-9583-775c24ad74e8.png)

---------

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-12 17:27:50 +00:00
rune
4b80813341
Support SSH for go get (#24664)
fix #12192 Support SSH for go get

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
Co-authored-by: mfk <mfk@hengwei.com.cn>
Co-authored-by: silverwind <me@silverwind.io>
2023-05-12 09:44:37 +00:00
oliverpool
8030614386
fix: release page for empty or non-existing target (#24470)
Fixes #24145

To solve the bug, I added a "computed" `TargetBehind` field to the
`Release` model, which indicates the target branch of a release.
This is particularly useful if the target branch was deleted in the
meantime (or is empty).

I also did a micro-optimization in `calReleaseNumCommitsBehind`. Instead
of checking that a branch exists and then call `GetBranchCommit`, I
immediately call `GetBranchCommit` and handle the `git.ErrNotExist`
error.

This optimization is covered by the added unit test.
2023-05-10 11:43:55 +08:00
Matthew Walowski
1dd83dbb91
Filters for GetAllCommits (#24568)
The `GetAllCommits` endpoint can be pretty slow, especially in repos
with a lot of commits. The issue is that it spends a lot of time
calculating information that may not be useful/needed by the user.

The `stat` param was previously added in #21337 to address this, by
allowing the user to disable the calculating stats for each commit. But
this has two issues:
1. The name `stat` is rather misleading, because disabling `stat`
disables the Stat **and** Files. This should be separated out into two
different params, because getting a list of affected files is much less
expensive than calculating the stats
2. There's still other costly information provided that the user may not
need, such as `Verification`

This PR, adds two parameters to the endpoint, `files` and `verification`
to allow the user to explicitly disable this information when listing
commits. The default behavior is true.
2023-05-09 09:06:05 +08:00
Nick
3d266dd0f3
In TestViewRepo2, convert computed timezones to local time (#24579)
This fixes up https://github.com/go-gitea/gitea/pull/10446; in that, the
*expected* timezone was changed to the local timezone, but the computed
timezone was left in UTC.

The result was this failure, when run on a non-UTC system:

```
	Diff:
    --- Expected
    +++ Actual
    @@ -5,3 +5,3 @@
       commitMsg: (string) (len=12) "init project",
    -  commitTime: (string) (len=29) "Wed, 14 Jun 2017 09:54:21 EDT"
    +  commitTime: (string) (len=29) "Wed, 14 Jun 2017 13:54:21 UTC"
      },
    @@ -11,3 +11,3 @@
       commitMsg: (string) (len=12) "init project",
    -  commitTime: (string) (len=29) "Wed, 14 Jun 2017 09:54:21 EDT"
    +  commitTime: (string) (len=29) "Wed, 14 Jun 2017 13:54:21 UTC"
      }
    Test:       	TestViewRepo2
```

I assume this was probably missed since the CI servers all run in UTC?

The Format() string "Mon, 02 Jan 2006 15:04:05 UTC" was incorrect: 'UTC'
isn't recognized as a variable placeholder, but was just being copied
verbatim. It should use 'MST' in order to command Format() to output the
attached timezone, which is what `time.RFC1123` has.
2023-05-08 21:07:41 +08:00
wxiaoguang
6f9c278559
Rewrite queue (#24505)
# ⚠️ Breaking

Many deprecated queue config options are removed (actually, they should
have been removed in 1.18/1.19).

If you see the fatal message when starting Gitea: "Please update your
app.ini to remove deprecated config options", please follow the error
messages to remove these options from your app.ini.

Example:

```
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]`
2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options
```

Many options in `[queue]` are are dropped, including:
`WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`,
`BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed
from app.ini.

# The problem

The old queue package has some legacy problems:

* complexity: I doubt few people could tell how it works.
* maintainability: Too many channels and mutex/cond are mixed together,
too many different structs/interfaces depends each other.
* stability: due to the complexity & maintainability, sometimes there
are strange bugs and difficult to debug, and some code doesn't have test
(indeed some code is difficult to test because a lot of things are mixed
together).
* general applicability: although it is called "queue", its behavior is
not a well-known queue.
* scalability: it doesn't seem easy to make it work with a cluster
without breaking its behaviors.

It came from some very old code to "avoid breaking", however, its
technical debt is too heavy now. It's a good time to introduce a better
"queue" package.

# The new queue package

It keeps using old config and concept as much as possible.

* It only contains two major kinds of concepts:
    * The "base queue": channel, levelqueue, redis
* They have the same abstraction, the same interface, and they are
tested by the same testing code.
* The "WokerPoolQueue", it uses the "base queue" to provide "worker
pool" function, calls the "handler" to process the data in the base
queue.
* The new code doesn't do "PushBack"
* Think about a queue with many workers, the "PushBack" can't guarantee
the order for re-queued unhandled items, so in new code it just does
"normal push"
* The new code doesn't do "pause/resume"
* The "pause/resume" was designed to handle some handler's failure: eg:
document indexer (elasticsearch) is down
* If a queue is paused for long time, either the producers blocks or the
new items are dropped.
* The new code doesn't do such "pause/resume" trick, it's not a common
queue's behavior and it doesn't help much.
* If there are unhandled items, the "push" function just blocks for a
few seconds and then re-queue them and retry.
* The new code doesn't do "worker booster"
* Gitea's queue's handlers are light functions, the cost is only the
go-routine, so it doesn't make sense to "boost" them.
* The new code only use "max worker number" to limit the concurrent
workers.
* The new "Push" never blocks forever
* Instead of creating more and more blocking goroutines, return an error
is more friendly to the server and to the end user.

There are more details in code comments: eg: the "Flush" problem, the
strange "code.index" hanging problem, the "immediate" queue problem.

Almost ready for review.

TODO:

* [x] add some necessary comments during review
* [x] add some more tests if necessary
* [x] update documents and config options
* [x] test max worker / active worker
* [x] re-run the CI tasks to see whether any test is flaky
* [x] improve the `handleOldLengthConfiguration` to provide more
friendly messages
* [x] fine tune default config values (eg: length?)

## Code coverage:

![image](https://user-images.githubusercontent.com/2114189/236620635-55576955-f95d-4810-b12f-879026a3afdf.png)
2023-05-08 19:49:59 +08:00
Matthew Walowski
ff5629268c
Pass 'not' to commit count (#24473)
Due to #24409 , we can now specify '--not' when getting all commits from
a repo to exclude commits from a different branch.

When I wrote that PR, I forgot to also update the code that counts the
number of commits in the repo. So now, if the --not option is used, it
may return too many commits, which can indicate that another page of
data is available when it is not.

This PR passes --not to the commands that count the number of commits in
a repo
2023-05-08 07:10:53 +00:00
Steve Russo
80765aab8c
Fix broken link in tests/e2e/README (#24576) 2023-05-08 00:52:11 +00:00
techknowlogick
4daf40505a
Sort users and orgs on explore by recency by default (#24279)
This gives more "freshness" to the explore page. So it's not just the
same X users on the explore page by default, now it matches the same
sort as the repos on the explore page.

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-05-06 22:04:55 +08:00
KN4CK3R
05209f0d1d
Add RPM registry (#23380)
Fixes #20751

This PR adds a RPM package registry. You can follow [this
tutorial](https://opensource.com/article/18/9/how-build-rpm-packages) to
build a *.rpm package for testing.

This functionality is similar to the Debian registry (#22854) and
therefore shares some methods. I marked this PR as blocked because it
should be merged after #22854.


![grafik](https://user-images.githubusercontent.com/1666336/223806549-d8784fd9-9d79-46a2-9ae2-f038594f636a.png)
2023-05-05 20:33:37 +00:00
Lunny Xiao
377a0a20f0
Merge setting.InitXXX into one function with options (#24389)
This PR will merge 3 Init functions on setting packages as 1 and
introduce an options struct.
2023-05-04 11:55:35 +08:00
silverwind
4a722c9a45
Make Issue/PR/projects more compact, misc CSS tweaks (#24459)
- Remove various horizontal dividers on repo pages that didn't provide
visual benefit
- Remove label/milestone pills on single issue/pr page
- Remove issue-related pill buttons on projects page
- Increase contrast of color-secondary on arc-green
- Improve notifications icon, make circle bigger
- Remove some inline styles
- Fix focus in issue/pr title edit and select all text on button click

### Issue and PR before and after

<img width="1249" alt="Screenshot 2023-05-01 at 11 44 22"
src="https://user-images.githubusercontent.com/115237/235436662-a708288e-84fb-4b2e-a5a2-3a1c17d28f6c.png">
<img width="1248" alt="Screenshot 2023-05-01 at 11 58 51"
src="https://user-images.githubusercontent.com/115237/235437992-f863e483-f3cc-4cc1-8204-fd223647a0c9.png">



### Projects before and after

<img width="1255" alt="Screenshot 2023-05-01 at 11 41 02"
src="https://user-images.githubusercontent.com/115237/235436433-0deb85d6-4e7d-4e74-847f-254cc70a0cf9.png">
<img width="1267" alt="Screenshot 2023-05-01 at 11 40 03"
src="https://user-images.githubusercontent.com/115237/235436431-715b13cb-f78c-4d86-b27a-9229f9738c5b.png">


### Releases before and after

<img width="1243" alt="Screenshot 2023-05-01 at 11 41 12"
src="https://user-images.githubusercontent.com/115237/235436457-b655ee6f-03b8-4595-8d8c-b15ea469e988.png">
<img width="1240" alt="Screenshot 2023-05-01 at 11 40 10"
src="https://user-images.githubusercontent.com/115237/235436456-05a2a0dd-7cbb-4f26-b0d3-4f667df4bb95.png">

### Misc

<img width="58" alt="Screenshot 2023-05-01 at 10 49 13"
src="https://user-images.githubusercontent.com/115237/235432494-936ce995-6e22-47bc-ab2d-c9e93d31987d.png">
<img width="57" alt="Screenshot 2023-05-01 at 18 57 08"
src="https://user-images.githubusercontent.com/115237/235492430-1d32cfe0-0f2c-467c-b2fa-925b27e30e0e.png">


Issue title edit and wrap:

<img width="1238" alt="Screenshot 2023-05-01 at 12 34 40"
src="https://user-images.githubusercontent.com/115237/235441407-d5067a57-e586-4865-a652-282e5944abb4.png">
<img width="1232" alt="Screenshot 2023-05-01 at 12 06 24"
src="https://user-images.githubusercontent.com/115237/235438710-1a543dda-220f-4d87-8f93-f1710c0695f0.png">

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-05-03 17:58:59 -04:00
KN4CK3R
723598b803
Implement Cargo HTTP index (#24452)
This implements the HTTP index
[RFC](https://rust-lang.github.io/rfcs/2789-sparse-index.html) for Cargo
registries.

Currently this is a preview feature and you need to use the nightly of
`cargo`:

`cargo +nightly -Z sparse-registry update`

See https://github.com/rust-lang/cargo/issues/9069 for more information.

---------

Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-03 16:58:43 -04:00
KN4CK3R
bf999e4069
Add Debian package registry (#24426)
Co-authored-by: @awkwardbunny

This PR adds a Debian package registry.
You can follow [this
tutorial](https://www.baeldung.com/linux/create-debian-package) to build
a *.deb package for testing.
Source packages are not supported at the moment and I did not find
documentation of the architecture "all" and how these packages should be
treated.


![grafik](https://user-images.githubusercontent.com/1666336/218126879-eb80a866-775c-4c8e-8529-5797203a64e6.png)

Part of #20751.

Revised copy of #22854.

---------

Co-authored-by: Brian Hong <brian@hongs.me>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-02 12:31:35 -04:00
silverwind
8f4dafcd4e
Rework header bar on issue, pull requests and milestone (#24420)
- Make search bar dynamic full width via flexbox
- Make all buttons `small` so font size is the same for all elements in
the header
- Remove primary color from search field, add SVG icon like on Code tab
- Fix button vertical padding being enlarged by SVG icons

[View diff without
whitespace](https://github.com/go-gitea/gitea/pull/24420/files?diff=unified&w=1)

<img width="1226" alt="Screenshot 2023-04-29 at 11 58 53"
src="https://user-images.githubusercontent.com/115237/235296851-74848267-664f-4c1f-b94c-a1b94196ff75.png">
<img width="1219" alt="Screenshot 2023-04-29 at 11 59 39"
src="https://user-images.githubusercontent.com/115237/235296852-bcfde5ed-8658-43c2-b7e5-3ad84611e76f.png">

Mobile:
<img width="437" alt="Screenshot 2023-04-29 at 11 59 52"
src="https://user-images.githubusercontent.com/115237/235296860-99263373-7b27-4540-868c-a93e70f281ca.png">
<img width="433" alt="Screenshot 2023-04-29 at 12 00 00"
src="https://user-images.githubusercontent.com/115237/235296862-6cf64317-a864-405a-a00f-b5ab620349f5.png">
2023-04-29 23:33:25 -04:00
6543
0bd05a9f1c
Add integration test for API raw content reference formats (#24388)
This pull request adds an integration test to validate the behavior of
raw content API's reference handling for all supported formats .

close  #24242

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-04-28 22:38:22 -04:00
Yarden Shoham
c0ddec8a2a
Revert "Add Debian package registry" (#24412)
Reverts go-gitea/gitea#22854
2023-04-28 18:06:41 -04:00
KN4CK3R
bf77e2163b
Add Debian package registry (#22854)
Co-authored-by: @awkwardbunny

This PR adds a Debian package registry. You can follow [this
tutorial](https://www.baeldung.com/linux/create-debian-package) to build
a *.deb package for testing. Source packages are not supported at the
moment and I did not find documentation of the architecture "all" and
how these packages should be treated.

---------

Co-authored-by: Brian Hong <brian@hongs.me>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2023-04-28 17:51:36 -04:00
Lunny Xiao
ecf1f2d3f6
Fix auth check bug (#24382)
Fix https://github.com/go-gitea/gitea/pull/24362/files#r1179095324

`getAuthenticatedMeta` has checked them, these code are duplicated one.
And the first invokation has a wrong permission check. `DownloadHandle`
should require read permission but not write.
2023-04-27 22:43:27 +02:00