Commit graph

1112 commits

Author SHA1 Message Date
Gusted d37d0773bc
[GITEA] Add download URL for executable files
- Consider executable files as a valid case when returning a downloadURL
for them. They are just regular files with the difference being the
executable permission bit being set.
- Not integration testing as it's not possible without adding adjusting
the existing repositories to have a executable file.
- Resolves https://codeberg.org/forgejo/forgejo/issues/1825

(cherry picked from commit ca32891d54)
(cherry picked from commit 72c9df8e45)
(cherry picked from commit 0eae22d429)
2023-12-18 16:39:20 +01:00
Gusted 9c74254d46
[GITEA] Allow user to select email for file operations in Web UI
- Add a dropdown to the web interface for changing files to select which
Email should be used for the commit. It only shows (and verifies) that a
activated mail can be used, while this isn't necessary, it's better to
have this already in place.
- Added integration testing.
- Resolves https://codeberg.org/forgejo/forgejo/issues/281

(cherry picked from commit 564e701f40)
(cherry picked from commit de8f2e03cc)
(cherry picked from commit 0182cff12e)
2023-12-18 16:39:20 +01:00
Antonin Delpeuch 6bbe75ecf8
[GITEA] Avoid conflicts of issue and PR numbers in GitLab migration (#1790)
Closes #1789.

The bug was due to the fact that GitLab does not guarantee that issue numbers are created sequentially: some identifiers can be skipped. Therefore, the new pull requests numbers should not be offset by the number of issues, but by the maximum issue number.

See for instance https://gitlab.com/troyengel/archbuild/-/issues/?sort=created_date&state=all&first_page_size=20, where there is only a singe issue with number "2".

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/1790
Co-authored-by: Antonin Delpeuch <antonin@delpeuch.eu>
Co-committed-by: Antonin Delpeuch <antonin@delpeuch.eu>
(cherry picked from commit 2c185c39fe)
(cherry picked from commit 8f68dc4c9c)
(cherry picked from commit 7e932b7fca)
2023-12-18 16:39:20 +01:00
Antonin Delpeuch f0746e648d
[GITEA] Enable mocked HTTP responses for GitLab migration test
Fix gitlab migration unit test

Closes #1837.

The differences in dates can be explained by commit e19b9653ea, which
changed the order in which "created_date" and "updated_date" are
considered.

(cherry picked from commit b0bba20aa4)

Mock HTTP requests in GitLab migration test

This introduces a new utility which can be added to other tests
making HTTP calls to a live service, to cache the responses of this
service in the repository.

(cherry picked from commit 52053b1389)

Enable mocked HTTP responses for GitLab migration test

(cherry picked from commit 19cefc4de2)

Simplify HTTP mocking utility in unit tests

Follow-up to https://codeberg.org/forgejo/forgejo/pulls/1841

(cherry picked from commit ca517c8bb4)
(cherry picked from commit b227e0dd6b)
(cherry picked from commit 6cc9d06556)
2023-12-18 16:39:20 +01:00
Gusted f25eeb7695
[GITEA] Use existing error functionality
- There's no need to use `github.com/pkg/errors` when the standard
library already has the functionality to wrap and create errors.

(cherry picked from commit 40f603a538)
(cherry picked from commit aa68a2753f)
(cherry picked from commit 48e252d739)
(cherry picked from commit cc6f40ccd2)
(cherry picked from commit 03c4b97358)
2023-12-18 16:39:20 +01:00
Gusted 0d0392f3a5
[GITEA] Drop sha256-simd in favor of stdlib
- In Go 1.21 the crypto/sha256 [got a massive
improvement](https://go.dev/doc/go1.21#crypto/sha256) by utilizing the
SHA instructions for AMD64 CPUs, which sha256-simd already was doing.
The performance is now on par and I think it's preferable to use the
standard library rather than a package when possible.

```
cpu: AMD Ryzen 5 3600X 6-Core Processor
                │  simd.txt   │               go.txt                │
                │   sec/op    │    sec/op     vs base               │
Hash/8Bytes-12    63.25n ± 1%    73.38n ± 1%  +16.02% (p=0.002 n=6)
Hash/64Bytes-12   98.73n ± 1%   105.30n ± 1%   +6.65% (p=0.002 n=6)
Hash/1K-12        567.2n ± 1%    572.8n ± 1%   +0.99% (p=0.002 n=6)
Hash/8K-12        4.062µ ± 1%    4.062µ ± 1%        ~ (p=0.396 n=6)
Hash/1M-12        512.1µ ± 0%    510.6µ ± 1%        ~ (p=0.485 n=6)
Hash/5M-12        2.556m ± 1%    2.564m ± 0%        ~ (p=0.093 n=6)
Hash/10M-12       5.112m ± 0%    5.127m ± 0%        ~ (p=0.093 n=6)
geomean           13.82µ         14.27µ        +3.28%

                │   simd.txt   │               go.txt                │
                │     B/s      │     B/s       vs base               │
Hash/8Bytes-12    120.6Mi ± 1%   104.0Mi ± 1%  -13.81% (p=0.002 n=6)
Hash/64Bytes-12   618.2Mi ± 1%   579.8Mi ± 1%   -6.22% (p=0.002 n=6)
Hash/1K-12        1.682Gi ± 1%   1.665Gi ± 1%   -0.98% (p=0.002 n=6)
Hash/8K-12        1.878Gi ± 1%   1.878Gi ± 1%        ~ (p=0.310 n=6)
Hash/1M-12        1.907Gi ± 0%   1.913Gi ± 1%        ~ (p=0.485 n=6)
Hash/5M-12        1.911Gi ± 1%   1.904Gi ± 0%        ~ (p=0.093 n=6)
Hash/10M-12       1.910Gi ± 0%   1.905Gi ± 0%        ~ (p=0.093 n=6)
geomean           1.066Gi        1.032Gi        -3.18%
```

(cherry picked from commit abd94ff5b5)
(cherry picked from commit 15e81637ab)

Conflicts:
	go.mod
	https://codeberg.org/forgejo/forgejo/pulls/1581
(cherry picked from commit 325d92917f)

Conflicts:
	modules/context/context_cookie.go
	https://codeberg.org/forgejo/forgejo/pulls/1617
(cherry picked from commit 358819e895)
(cherry picked from commit 362fd7aae1)
(cherry picked from commit 4f64ee294e)
(cherry picked from commit 4bde77f7b1)
(cherry picked from commit 1311e30a81)
(cherry picked from commit 57b69e334c)
(cherry picked from commit 52dc892fad)
(cherry picked from commit 77f54f4187)
2023-12-18 16:39:19 +01:00
Gusted 9b17353f85
[GITEA] Skip unsupported code comment
- If there's a code comment that's received during the migration that
contains no diffhunk, skip it. This either means it was commenting on
old diffhunk or it's just a general codecomment. Forgejo supports
neither of such type of code comment.
- Resolves https://codeberg.org/forgejo/forgejo/issues/1407

(cherry picked from commit ae463c7c55)
(cherry picked from commit bf48f02a86)
(cherry picked from commit 10c3f102fa)
(cherry picked from commit 828b4cc10c)
(cherry picked from commit 6427fa65b6)
(cherry picked from commit 5b7a43c43f)
(cherry picked from commit 4eef0fce72)
(cherry picked from commit a46192a4a6)
(cherry picked from commit 107a9b8233)
(cherry picked from commit 308251fc48)
(cherry picked from commit 017c4a53c5)
(cherry picked from commit 4534a3393b)
(cherry picked from commit 74e0c1663d)
2023-12-18 16:39:19 +01:00
Aravinth Manivannan d48b84f623
[GITEA] notifies admins on new user registration
Sends email with information on the new user (time of creation and time of last sign-in) and a link to manage the new user from the admin panel

closes: https://codeberg.org/forgejo/forgejo/issues/480

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/1371
Co-authored-by: Aravinth Manivannan <realaravinth@batsense.net>
Co-committed-by: Aravinth Manivannan <realaravinth@batsense.net>
(cherry picked from commit c721aa828b)
(cherry picked from commit 6487efcb9d)

Conflicts:
	modules/notification/base/notifier.go
	modules/notification/base/null.go
	modules/notification/notification.go
	https://codeberg.org/forgejo/forgejo/pulls/1422
(cherry picked from commit 7ea66ee1c5)

Conflicts:
	services/notify/notifier.go
	services/notify/notify.go
	services/notify/null.go
	https://codeberg.org/forgejo/forgejo/pulls/1469
(cherry picked from commit 7d2d997011)
(cherry picked from commit 435a54f140)
(cherry picked from commit 8ec7b3e448)

[GITEA] notifies admins on new user registration (squash) performance bottleneck

Refs: https://codeberg.org/forgejo/forgejo/issues/1479
(cherry picked from commit 97ac9147ff)
(cherry picked from commit 19f295c16b)
(cherry picked from commit 3367dcb2cf)

[GITEA] notifies admins on new user registration (squash) cosmetic changes

Co-authored-by: delvh <dev.lh@web.de>
(cherry picked from commit 9f1670e040)
(cherry picked from commit de5bb2a224)
(cherry picked from commit 8f8e52f31a)
(cherry picked from commit e0d5130312)
(cherry picked from commit f1288d6d9b)
(cherry picked from commit 1db4736fd7)
(cherry picked from commit e8dcbb6cd6)
(cherry picked from commit 09625d6476)

[GITEA] notifies admins on new user registration (squash) ctx.Locale

(cherry picked from commit dab7212fad)
(cherry picked from commit 9b7bbae8c4)
(cherry picked from commit f750b71d3d)
(cherry picked from commit f79af36679)
(cherry picked from commit e76eee334e)

[GITEA] notifies admins on new user registration (squash) fix locale

(cherry picked from commit 54cd100d8d)
(cherry picked from commit 053dbd3d50)

[GITEA] notifies admins on new user registration (squash) fix URL

1. Use absolute URL in the admin panel link sent on new registrations
2. Include absolute URL of the newly signed-up user's profile.

New email looks like this:

<details><summary>Please click to expand</summary>

```
--153937b1864f158f4fd145c4b5d4a513568681dd489021dd466a8ad7b770
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=UTF-8

User Information: @realaravinth ( http://localhost:3000/realaravinth )
----------------------------------------------------------------------

* Created: 2023-12-13 19:36:50 +05:30

Please click here ( http://localhost:3000/admin/users/9 ) to manage the use=
r from the admin panel.
--153937b1864f158f4fd145c4b5d4a513568681dd489021dd466a8ad7b770
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset=UTF-8

<!DOCTYPE html>
<html>
<head>
	<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dutf-8">
	<title>New user realaravinth just signed up</title>

	<style>
		blockquote { padding-left: 1em; margin: 1em 0; border-left: 1px solid gre=
y; color: #777}
		.footer { font-size:small; color:#666;}
	</style>

</head>

<body>
	<ul>
		<h3>User Information: <a href=3D"http://localhost:3000/realaravinth">@rea=
laravinth</a></h3>
		<li>Created: <relative-time format=3D"datetime" weekday=3D"" year=3D"nume=
ric" month=3D"short" day=3D"numeric" hour=3D"numeric" minute=3D"numeric" se=
cond=3D"numeric" datetime=3D"2023-12-13T19:36:50+05:30">2023-12-13 19:36:50=
 +05:30</relative-time></li>
	</ul>
	<p> Please <a href=3D"http://localhost:3000/admin/users/9" rel=3D"nofollow=
">click here</a> to manage the user from the admin panel. </p>
</body>
</html>

--153937b1864f158f4fd145c4b5d4a513568681dd489021dd466a8ad7b770--
```

</details>

fixes: https://codeberg.org/forgejo/forgejo/issues/1927
Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/1940
Reviewed-by: Earl Warren <earl-warren@noreply.codeberg.org>
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Aravinth Manivannan <realaravinth@batsense.net>
Co-committed-by: Aravinth Manivannan <realaravinth@batsense.net>
(cherry picked from commit b8d764e36a)
2023-12-18 16:39:16 +01:00
Gusted 64f053f383
[SECURITY] Rework long-term authentication
- This is a 'front-port' of the already existing patch on v1.21 and
v1.20, but applied on top of what Gitea has done to rework the LTA
mechanism. Forgejo will stick with the reworked mechanism by the Forgejo
Security team for the time being. The removal of legacy code (AES-GCM) has been
left out.
- The current architecture is inherently insecure, because you can
construct the 'secret' cookie value with values that are available in
the database. Thus provides zero protection when a database is
dumped/leaked.
- This patch implements a new architecture that's inspired from: [Paragonie Initiative](https://paragonie.com/blog/2015/04/secure-authentication-php-with-long-term-persistence#secure-remember-me-cookies).
- Integration testing is added to ensure the new mechanism works.
- Removes a setting, because it's not used anymore.

(cherry picked from commit e3d6622a63)
(cherry picked from commit fef1a6dac5)
(cherry picked from commit b0c5165145)
(cherry picked from commit 7ad51b9f8d)
2023-12-18 15:12:41 +01:00
Earl Warren 2f0b4a8f61
[FEAT] allow setting the update date on issues and comments
This field adds the possibility to set the update date when modifying
an issue through the API.

A 'NoAutoDate' in-memory field is added in the Issue struct.
If the update_at field is set, NoAutoDate is set to true and the
Issue's UpdatedUnix field is filled.

That information is passed down to the functions that actually updates
the database, which have been modified to not auto update dates if
requested.

A guard is added to the 'EditIssue' API call, to checks that the
udpate_at date is between the issue's creation date and the current
date (to avoid 'malicious' changes). It also limits the new feature
to project's owners and admins.

(cherry picked from commit c524d33402)

Add a SetIssueUpdateDate() function in services/issue.go

That function is used by some API calls to set the NoAutoDate and
UpdatedUnix fields of an Issue if an updated_at date is provided.

(cherry picked from commit f061caa655)

Add an updated_at field to the API calls related to Issue's Labels.

The update date is applied to the issue's comment created to inform
about the modification of the issue's labels.

(cherry picked from commit ea36cf80f5)

Add an updated_at field to the API call for issue's attachment creation

The update date is applied to the issue's comment created to inform
about the modification of the issue's content, and is set as the
asset creation date.

(cherry picked from commit 96150971ca)

Checking Issue changes, with and without providing an updated_at date

Those unit tests are added:

- TestAPIEditIssueWithAutoDate
- TestAPIEditIssueWithNoAutoDate

- TestAPIAddIssueLabelsWithAutoDate
- TestAPIAddIssueLabelsWithNoAutoDate

- TestAPICreateIssueAttachmentWithAutoDate
- TestAPICreateIssueAttachmentWithNoAutoDate

(cherry picked from commit 4926a5d7a2)

Add an updated_at field to the API call for issue's comment creation

The update date is used as the comment creation date, and is applied to
the issue as the update creation date.

(cherry picked from commit 76c8faecdc)

Add an updated_at field to the API call for issue's comment edition

The update date is used as the comment update date, and is applied to
the issue as an update date.

(cherry picked from commit cf787ad7fd)

Add an updated_at field to the API call for comment's attachment creation

The update date is applied to the comment, and is set as the asset
creation date.

(cherry picked from commit 1e4ff424d3)

Checking Comment changes, with and without providing an updated_at date

Those unit tests are added:

- TestAPICreateCommentWithAutoDate
- TestAPICreateCommentWithNoAutoDate

- TestAPIEditCommentWithAutoDate
- TestAPIEditCommentWithNoAutoDate

- TestAPICreateCommentAttachmentWithAutoDate
- TestAPICreateCommentAttachmentWithNoAutoDate

(cherry picked from commit da932152f1)

Pettier code to set the update time of comments

Now uses sess.AllCols().NoAutoToime().SetExpr("updated_unix", ...)

XORM is smart enough to compose one single SQL UPDATE which all
columns + updated_unix.

(cherry picked from commit 1f6a42808d)

Issue edition: Keep the max of the milestone and issue update dates.

When editing an issue via the API, an updated_at date can be provided.
If the EditIssue call changes the issue's milestone, the milestone's
update date is to be changed accordingly, but only with a greater
value.

This ensures that a milestone's update date is the max of all issue's
update dates.

(cherry picked from commit 8f22ea182e)

Rewrite the 'AutoDate' tests using subtests

Also add a test to check the permissions to set a date, and a test
to check update dates on milestones.

The tests related to 'AutoDate' are:
- TestAPIEditIssueAutoDate
- TestAPIAddIssueLabelsAutoDate
- TestAPIEditIssueMilestoneAutoDate
- TestAPICreateIssueAttachmentAutoDate
- TestAPICreateCommentAutoDate
- TestAPIEditCommentWithDate
- TestAPICreateCommentAttachmentAutoDate

(cherry picked from commit 961fd13c55)
(cherry picked from commit d52f4eea44)
(cherry picked from commit 3540ea2a43)

Conflicts:
	services/issue/issue.go
	https://codeberg.org/forgejo/forgejo/pulls/1415
(cherry picked from commit 56720ade00)

Conflicts:
	routers/api/v1/repo/issue_label.go
	https://codeberg.org/forgejo/forgejo/pulls/1462
(cherry picked from commit 47c78927d6)
(cherry picked from commit 2030f3b965)
(cherry picked from commit f02aeb7698)

Conflicts:
	routers/api/v1/repo/issue_attachment.go
	routers/api/v1/repo/issue_comment_attachment.go
	https://codeberg.org/forgejo/forgejo/pulls/1575
(cherry picked from commit d072525b35)
(cherry picked from commit 8424d0ab3d)
(cherry picked from commit 5cc62caec7)
(cherry picked from commit d6300d5dcd)

[FEAT] allow setting the update date on issues and comments (squash) apply the 'update_at' value to the cross-ref comments (#1676)

[this is a follow-up to PR #764]

When a comment of issue A referencing issue B is added with a forced 'updated_at' date, that date has to be applied to the comment created in issue B.

-----

Comment:

While trying my 'RoundUp migration script', I found that this case was forgotten in PR #764 - my apologies...

I'll try to write a functional test, base on models/issues/issue_xref_test.go

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/1676
Co-authored-by: fluzz <fluzz@freedroid.org>
Co-committed-by: fluzz <fluzz@freedroid.org>
(cherry picked from commit ac4f727f63)
(cherry picked from commit 5110476ee9)
(cherry picked from commit 77ba6be1da)
(cherry picked from commit 9c8337b5c4)
(cherry picked from commit 1d689eb686)
(cherry picked from commit 511c519c87)
2023-12-18 15:12:41 +01:00
Earl Warren 7e7492b996
[UPGRADE] run sanity checks before the database is upgraded
https://codeberg.org/forgejo/forgejo/pulls/1543
(cherry picked from commit fbb71f464b)
(cherry picked from commit 650f849c84)
(cherry picked from commit 25e0d33dfe)
(cherry picked from commit 222b70e504)
(cherry picked from commit 4be04627a7)
(cherry picked from commit 64fcb25de3)
(cherry picked from commit 740943d6b0)
(cherry picked from commit 3c096abf11)
(cherry picked from commit da04dcb099)
(cherry picked from commit b881d83f1c)
2023-12-18 15:12:41 +01:00
Earl Warren 3b1ebd95b9
[UPGRADE] add sanity checks for [storage*]
Refs: https://forgejo.org/2023-08-release-v1-20-3-0/
(cherry picked from commit a266dd0ce3)
(cherry picked from commit b9eb5eccd8)
(cherry picked from commit 7fc2028ede)
(cherry picked from commit 0c988e6120)
(cherry picked from commit 7ba05e8c2b)
(cherry picked from commit 2ed5068abe)
(cherry picked from commit 353913a26d)
(cherry picked from commit 4e63a01a8b)
(cherry picked from commit 99f612aed3)
(cherry picked from commit b4fe189cae)
(cherry picked from commit bd35e3b7bc)
(cherry picked from commit f59d9f7088)
(cherry picked from commit 0b2a93e044)
(cherry picked from commit 8c5d8bfea0)
(cherry picked from commit 2817ce027c)
(cherry picked from commit 162056cbcf)
(cherry picked from commit 07152a0ba6)
(cherry picked from commit 72e3777915)
2023-12-18 15:10:54 +01:00
Earl Warren 8ba6a4c9db
[UPGRADE] run sanity checks before the database is upgraded
(cherry picked from commit 69741e4e66)
(cherry picked from commit 2a3c7b09cb)
(cherry picked from commit a1554c1168)
(cherry picked from commit edae2c6d2d)
(cherry picked from commit 49737cf009)
(cherry picked from commit ec53704c34)
(cherry picked from commit 7a1c5c0f32)
(cherry picked from commit e658c20c0f)
(cherry picked from commit baf575468f)
(cherry picked from commit 40cb14eff4)
(cherry picked from commit 25ab4d0713)
(cherry picked from commit 5a29005215)
(cherry picked from commit fef1260e99)
(cherry picked from commit eadbbb1afe)
(cherry picked from commit db22d61eb4)
(cherry picked from commit 9d3b0be39a)
(cherry picked from commit b3fa3c1292)
(cherry picked from commit c8300d4fe2)
2023-12-18 15:10:54 +01:00
Earl Warren 301e5d9f27
[TESTS] oauth2: make it possible to use an alternate http.Client
(cherry picked from commit aea4ab25a9)
(cherry picked from commit d386b212c4)
(cherry picked from commit c4935f08ad)
(cherry picked from commit dc6ca7cd25)
(cherry picked from commit 25296d5a3c)
(cherry picked from commit 3d54c64c5f)
(cherry picked from commit 6ece0b9d01)
(cherry picked from commit 3b39962033)
(cherry picked from commit 5e2167cd03)
(cherry picked from commit e676d7b265)
(cherry picked from commit 9cd258e865)
(cherry picked from commit 0a8d58c159)
(cherry picked from commit b66d06823a)
(cherry picked from commit 4fbe2a0047)
(cherry picked from commit a225e0c9b4)
(cherry picked from commit 21b670b927)
(cherry picked from commit d586e335d5)
(cherry picked from commit 7ab21549a3)
(cherry picked from commit eb3235039f)
(cherry picked from commit 02b16875cf)
(cherry picked from commit 130a1617aa)
(cherry picked from commit e9ddf75e6b)
(cherry picked from commit ea3c79817f)
2023-12-18 15:10:54 +01:00
Gusted a064065cb9
[GITEA] Remove redundant syncBranchToDB
- The transaction in combination with Git push was causing deadlocks if
you had the `push_update` queue set to `immediate`. This was the root
cause of slow integration tests in CI.
- Remove the sync branch code as this is already being done in the Git
post-receive hook.
- Add tests to proof the branch models are in sync even with this code
removed.

(cherry picked from commit 90110e1f44)
2023-12-18 14:39:08 +01:00
wxiaoguang 20929edc99
Add option to disable ambiguous unicode characters detection (#28454)
* Close #24483
* Close #28123
* Close #23682
* Close #23149

(maybe more)
2023-12-17 14:38:54 +00:00
Lunny Xiao 408a484224
Adjust object format interface (#28469)
- Remove `ObjectFormatID`
- Remove function `ObjectFormatFromID`.
- Use `Sha1ObjectFormat` directly but not a pointer because it's an
empty struct.
- Store `ObjectFormatName` in `repository` struct
2023-12-17 11:56:08 +00:00
Adam Majer cbf923e87b
Abstract hash function usage (#28138)
Refactor Hash interfaces and centralize hash function. This will allow
easier introduction of different hash function later on.

This forms the "no-op" part of the SHA256 enablement patch.
2023-12-13 21:02:00 +00:00
KN4CK3R ff5106d700
Fix possible nil pointer access (#28428)
There could be a nil pointer exception if the file is not found because
that specific error is suppressed but not handled.
2023-12-12 13:51:33 +00:00
Jack Hay 4e879fed90
Deprecate query string auth tokens (#28390)
## Changes
- Add deprecation warning to `Token` and `AccessToken` authentication
methods in swagger.
- Add deprecation warning header to API response. Example: 
  ```
  HTTP/1.1 200 OK
  ...
  Warning: token and access_token API authentication is deprecated
  ...
  ```
- Add setting `DISABLE_QUERY_AUTH_TOKEN` to reject query string auth
tokens entirely. Default is `false`

## Next steps
- `DISABLE_QUERY_AUTH_TOKEN` should be true in a subsequent release and
the methods should be removed in swagger
- `DISABLE_QUERY_AUTH_TOKEN` should be removed and the implementation of
the auth methods in question should be removed

## Open questions
- Should there be further changes to the swagger documentation?
Deprecation is not yet supported for security definitions (coming in
[OpenAPI Spec version
3.2.0](https://github.com/OAI/OpenAPI-Specification/issues/2506))
- Should the API router logger sanitize urls that use `token` or
`access_token`? (This is obviously an insufficient solution on its own)

---------

Co-authored-by: delvh <dev.lh@web.de>
2023-12-12 03:48:53 +00:00
Lunny Xiao 537fa69962
Second part of refactor db.Find (#28194)
Continue of #27798 and move more functions to `db.Find` and `db.Count`.
2023-12-11 16:56:48 +08:00
Earl Warren fab73e4a3a
Actually recover from a panic in cron task (#28409)
- Currently there's code to recover gracefully from panics that happen
within the execution of cron tasks. However this recover code wasn't
being run, because `RunWithShutdownContext` also contains code to
recover from any panic and then gracefully shutdown Forgejo. Because
`RunWithShutdownContext` registers that code as last, that would get run
first which in this case is not behavior that we want.
- Move the recover code to inside the function, so that is run first
before `RunWithShutdownContext`'s recover code (which is now a noop).

Fixes: https://codeberg.org/forgejo/forgejo/issues/1910

Co-authored-by: Gusted <postmaster@gusted.xyz>
2023-12-10 20:15:06 +00:00
Lunny Xiao aeb383025f
Also sync DB branches on push if necessary (#28361)
Fix #28056 

This PR will check whether the repo has zero branch when pushing a
branch. If that, it means this repository hasn't been synced.

The reason caused that is after user upgrade from v1.20 -> v1.21, he
just push branches without visit the repository user interface. Because
all repositories routers will check whether a branches sync is necessary
but push has not such check.

For every repository, it has two states, synced or not synced. If there
is zero branch for a repository, then it will be assumed as non-sync
state. Otherwise, it's synced state. So if we think it's synced, we just
need to update branch/insert new branch. Otherwise do a full sync. So
that, for every push, there will be almost no extra load added. It's
high performance than yours.

For the implementation, we in fact will try to update the branch first,
if updated success with affect records > 0, then all are done. Because
that means the branch has been in the database. If no record is
affected, that means the branch does not exist in database. So there are
two possibilities. One is this is a new branch, then we just need to
insert the record. Another is the branches haven't been synced, then we
need to sync all the branches into database.
2023-12-09 13:30:56 +00:00
Lunny Xiao dd30d9d5c0
Remove GetByBean method because sometimes it's danger when query condition parameter is zero and also introduce new generic methods (#28220)
The function `GetByBean` has an obvious defect that when the fields are
empty values, it will be ignored. Then users will get a wrong result
which is possibly used to make a security problem.

To avoid the possibility, this PR removed function `GetByBean` and all
references.
And some new generic functions have been introduced to be used.

The recommand usage like below.

```go
// if query an object according id
obj, err := db.GetByID[Object](ctx, id)
// query with other conditions
obj, err := db.Get[Object](ctx, builder.Eq{"a": a, "b":b})
```
2023-12-07 15:27:36 +08:00
Nanguan Lin 49b98e45bc
Fix migration panic due to an empty review comment diff (#28334)
Fix #28328 
```
func (p *PullRequestComment) GetDiffHunk() string {
	if p == nil || p.DiffHunk == nil {
		return ""
	}
	return *p.DiffHunk
}
```
This function in the package `go-github` may return an empty diff. When
it's empty, the following code will panic because it access `ss[1]`

ec1feedbf5/services/migrations/gitea_uploader.go (L861-L867)

ec1feedbf5/modules/git/diff.go (L97-L101)
2023-12-05 08:29:43 +00:00
KN4CK3R dfa77ac020
Fix RPM/Debian signature key creation (#28352)
Fixes #28324

The name parameter can't contain some characters
(https://github.com/keybase/go-crypto/blob/master/openpgp/keys.go#L680)
but is optional. Therefore just use an empty string.
2023-12-05 00:48:01 +00:00
Brecht Van Lommel bffbf08f26
Fix missing issue search index update when changing status (#28325)
Changing an issue status, assignee, labels or milestone without also
adding a comment would not update the index, resulting in wrong search
results.
2023-12-03 12:22:44 +01:00
Earl Warren 1bfcdeef4c
Make pushUpdate error verbose (#28263)
- Push commits updates are run in a queue and updates can come from less
traceable places such as Git over SSH, therefor add more information
about on which repository the pushUpdate failed.

Refs: https://codeberg.org/forgejo/forgejo/pulls/1723

(cherry picked from commit 37ab946039)

Co-authored-by: Gusted <postmaster@gusted.xyz>
2023-11-30 00:46:51 +00:00
Earl Warren 64cd6e8df5
Fix required error for token name (#28267)
- Say to the binding middleware which locale should be used for the
required error.
- Resolves https://codeberg.org/forgejo/forgejo/issues/1683

(cherry picked from commit 5a2d796612)

Co-authored-by: Gusted <postmaster@gusted.xyz>
2023-11-29 03:47:05 +00:00
Lunny Xiao 882e502327
Fix comment permissions (#28213)
This PR will fix some missed checks for private repositories' data on
web routes and API routes.
2023-11-25 17:21:21 +00:00
Lunny Xiao df1e7d0067
Use db.Find instead of writing methods for every object (#28084)
For those simple objects, it's unnecessary to write the find and count
methods again and again.
2023-11-24 03:49:41 +00:00
Denys Konovalov 816e46ee7c
add skip ci functionality (#28075)
Adds the possibility to skip workflow execution if the commit message
contains a string like [skip ci] or similar.

The default strings are the same as on GitHub, users can also set custom
ones in app.ini

Reference:
https://docs.github.com/en/actions/managing-workflow-runs/skipping-workflow-runs

Close #28020
2023-11-18 13:37:08 +02:00
KN4CK3R 58f5fa6536
Fix Matrix and MSTeams nil dereference (#28089)
Fixes #28088 
Fixes #28094

Added missing tests.

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-11-17 12:17:33 +01:00
Lunny Xiao f65977df3a
Revert "packages: Calculate package size quota using package creator ID instead of owner ID (#28007)" (#28049)
This reverts commit #28007 60522fc96f.
2023-11-14 16:03:56 +01:00
Danila Fominykh 60522fc96f
packages: Calculate package size quota using package creator ID instead of owner ID (#28007)
Changed behavior to calculate package quota limit using package `creator
ID` instead of `owner ID`.

Currently, users are allowed to create an unlimited number of
organizations, each of which has its own package limit quota, resulting
in the ability for users to have unlimited package space in different
organization scopes. This fix will calculate package quota based on
`package version creator ID` instead of `package version owner ID`
(which might be organization), so that users are not allowed to take
more space than configured package settings.

Also, there is a side case in which users can publish packages to a
specific package version, initially published by different user, taking
that user package size quota. Version in fix should be better because
the total amount of space is limited to the quota for users sharing the
same organization scope.
2023-11-14 00:38:50 +01:00
KN4CK3R 3081e7e153
Fix missing mail reply address (#27997)
Fixes https://codeberg.org/forgejo/forgejo/issues/1458

Some mails such as issue creation mails are missing the reply-to-comment
address. This PR fixes that and specifies which comment types should get
a reply-possibility.
2023-11-13 03:20:34 +00:00
KN4CK3R 4f4fea734c
Unify two factor check (#27915)
Fixes #27819

We have support for two factor logins with the normal web login and with
basic auth. For basic auth the two factor check was implemented at three
different places and you need to know that this check is necessary. This
PR moves the check into the basic auth itself.
2023-11-06 08:22:39 +00:00
Earl Warren 59f9ef9fee
Remove action runners on user deletion (#27902)
- On user deletion, delete action runners that the user has created.
- Add a database consistency check to remove action runners that have
nonexistent belonging owner.
- Resolves https://codeberg.org/forgejo/forgejo/issues/1720

(cherry picked from commit 009ca7223d)

Co-authored-by: Gusted <postmaster@gusted.xyz>
2023-11-05 13:48:32 +01:00
Zettat123 ae396ac7c0
Fix DownloadFunc when migrating releases (#27887)
We should not use `asset.ID` in DownloadFunc because DownloadFunc is a
closure.

1bf5527eac/services/migrations/gitea_downloader.go (L284-L295)

A similar bug when migrating from GitHub has been fixed in #14703. This
PR fixes the bug when migrating from Gitea and GitLab.
2023-11-03 15:56:53 +08:00
Lunny Xiao 1bf5527eac
Refactor Find Sources and fix bug when view a user who belongs to an unactive auth source (#27798)
The steps to reproduce it.

First, create a new oauth2 source.
Then, a user login with this oauth2 source.
Disable the oauth2 source.
Visit users -> settings -> security, 500 will be displayed.
This is because this page only load active Oauth2 sources but not all
Oauth2 sources.
2023-11-03 01:41:00 +00:00
wxiaoguang a4b242ae7a
Clean up template locale usage (#27856)
After many refactoring PRs for the "locale" and "template context
function", now the ".locale" is not needed for web templates any more.

This PR does a clean up for:

1. Remove `ctx.Data["locale"]` for web context.
2. Use `ctx.Locale` in `500.tmpl`, for consistency.
3. Add a test check for `500 page` locale usage.
4. Remove the `Str2html` and `DotEscape` from mail template context
data, they are copy&paste errors introduced by #19169 and #16200 . These
functions are template functions (provided by the common renderer), but
not template data variables.
5. Make email `SendAsync` function mockable (I was planning to add more
tests but it would make this PR much too complex, so the tests could be
done in another PR)
2023-10-31 22:11:48 +08:00
Lunny Xiao 16d15ce087
Fix package webhook (#27839)
Fix #23742

---------

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-10-31 04:43:38 +00:00
KN4CK3R b36e2ca419
List all Debian package versions in Packages (#27786)
Closes #27783

This PR lists all and not only the latest package versions in the
`Packages` index.
2023-10-29 14:14:47 +00:00
Sebastian Brückner 02dae3f84b
Fix merge base commit for fast-forwarded GitLab PRs (#27825)
Due to a bug in the GitLab API, the diff_refs field is populated in the
response when fetching an individual merge request, but not when
fetching a list of them. That field is used to populate the merge base
commit SHA.
While there is detection for the merge base even when not populated by
the downloader, that detection is not flawless. Specifically, when a
GitLab merge request has a single commit, and gets merged with the
squash strategy, the base branch will be fast-forwarded instead of a
separate squash or merge commit being created. The merge base detection
attempts to find the last commit on the base branch that is also on the
PR branch, but in the fast-forward case that is the PR's only commit.
Assuming the head commit is also the merge base results in the import of
a PR with 0 commits and no diff.

This PR uses the individual merge request endpoint to fetch merge
request data with the diff_refs field. With its data, the base merge
commit can be properly set, which—by not relying on the detection
mentioned above—correctly imports PRs that were "merged" by
fast-forwarding the base branch.

ref: https://gitlab.com/gitlab-org/gitlab/-/issues/29620
2023-10-29 21:29:33 +08:00
Sebastian Brückner 8faa38568b
Use GitLab's squash_commit_sha when available (#27824)
Before this PR, the PR migration code populates Gitea's MergedCommitID
field by using GitLab's merge_commit_sha field. However, that field is
only populated when the PR was merged using a merge strategy. When a
squash strategy is used, squash_commit_sha is populated instead.

Given that Gitea does not keep track of merge and squash commits
separately, this PR simply populates Gitea's MergedCommitID by using
whichever field is present in the GitLab API response.
2023-10-29 00:54:58 +00:00
KN4CK3R ab3f6c1bac
Close all hashed buffers (#27787)
Add missing `.Close()` calls. The current code does not delete the
temporary files if the data grows over 32mb.
2023-10-25 21:24:24 +02:00
merlleu 796ff26e0e
Do not force creation of _cargo-index repo on publish (#27266)
Hello there,
Cargo Index over HTTP is now prefered over git for package updates: we
should not force users who do not need the GIT repo to have the repo
created/updated on each publish (it can still be created in the packages
settings).

The current behavior when publishing is to check if the repo exist and
create it on the fly if not, then update it's content.
Cargo HTTP Index does not rely on the repo itself so this will be
useless for everyone not using the git protocol for cargo registry.

This PR only disable the creation on the fly of the repo when publishing
a crate.

This is linked to #26844 (error 500 when trying to publish a crate if
user is missing write access to the repo) because it's now optional.

---------

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-10-24 03:26:38 +00:00
silverwind ce83609ff6
Upgrade to golangci-lint@v1.55.0 (#27756)
https://github.com/golangci/golangci-lint/releases/tag/v1.55.0
2023-10-24 02:54:59 +00:00
wxiaoguang f3956fcb28
Fix incorrect ctx usage in defer function (#27740) 2023-10-22 14:12:27 +00:00
6543 e83f2cbbac
Add doctor dbconsistency fix to delete repos with no owner (#27290)
to address #27273
replace #24873
2023-10-19 15:20:52 +02:00