Commit graph

43 commits

Author SHA1 Message Date
Gusted 2b28bf826e
[GITEA] Make atomic ssh keys replacement robust
- After stumbling upon https://github.com/golang/go/issues/22397 and
reading the implementations I realized that Forgejo code doesn't have
`Sync()` and it doesn't properly error handle the `Close` function.
- (likely) Resolves https://codeberg.org/forgejo/forgejo/issues/1446

(cherry picked from commit 0efcb334c2)
(cherry picked from commit 04ef02c0dd)
(cherry picked from commit 85f2065c9b)
(cherry picked from commit 8d36b5cce6)
(cherry picked from commit 378dc30fb5)
2023-10-30 14:37:04 +01:00
Lunny Xiao cddf245c12
Replace more db.DefaultContext (#27628)
Target #27065
2023-10-15 17:46:06 +02:00
JakobDev 76a85a4ce9
Final round of db.DefaultContext refactor (#27587)
Last part of #27065
2023-10-14 08:37:24 +00:00
Nanguan Lin dc04044716
Replace assert.Fail with assert.FailNow (#27578)
assert.Fail() will continue to execute the code while assert.FailNow()
not. I thought those uses of assert.Fail() should exit immediately.
PS: perhaps it's a good idea to use
[require](https://pkg.go.dev/github.com/stretchr/testify/require)
somewhere because the assert package's default behavior does not exit
when an error occurs, which makes it difficult to find the root error
reason.
2023-10-11 11:02:24 +00:00
JakobDev ebe803e514
Penultimate round of db.DefaultContext refactor (#27414)
Part of #27065

---------

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-10-11 04:24:07 +00:00
Sebastian Grabowski 3c131307ac
Respect SSH.KeygenPath option when calculating ssh key fingerprints (#27536)
Fixes #27535
2023-10-10 00:01:46 +00:00
Lunny Xiao 673cf6af76
make writing main test easier (#27270)
This PR removed `unittest.MainTest` the second parameter
`TestOptions.GiteaRoot`. Now it detects the root directory by current
working directory.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-09-28 01:38:53 +00:00
JakobDev 7047df36d4
Another round of db.DefaultContext refactor (#27103)
Part of #27065

---------

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2023-09-25 13:17:37 +00:00
JakobDev 76659b1114
Reduce usage of db.DefaultContext (#27073)
Part of #27065

This reduces the usage of `db.DefaultContext`. I think I've got enough
files for the first PR. When this is merged, I will continue working on
this.

Considering how many files this PR affect, I hope it won't take to long
to merge, so I don't end up in the merge conflict hell.

---------

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-09-14 17:09:32 +00:00
silverwind 887a683af9
Update tool dependencies, lock govulncheck and actionlint (#25655)
- Update all tool dependencies
- Lock `govulncheck` and `actionlint` to their latest tags

---------

Co-authored-by: 6543 <m.huber@kithara.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2023-07-09 11:58:06 +00:00
6543 8995046110
Less naked returns (#25713)
just a step towards  #25655

and some related refactoring
2023-07-07 05:31:56 +00:00
silverwind 88f835192d
Replace interface{} with any (#25686)
Result of running `perl -p -i -e 's#interface\{\}#any#g' **/*` and `make fmt`.

Basically the same [as golang did](2580d0e08d).
2023-07-04 18:36:08 +00:00
Lunny Xiao 377a0a20f0
Merge setting.InitXXX into one function with options (#24389)
This PR will merge 3 Init functions on setting packages as 1 and
introduce an options struct.
2023-05-04 11:55:35 +08:00
Leon Busch-George 7a8a4f5432
Prefer native parser for SSH public key parsing (#23798)
Without this patch, the setting SSH.StartBuiltinServer decides whether
the native (Go) implementation is used rather than calling 'ssh-keygen'.
It's possible for 'using ssh-keygen' and 'using the built-in server' to
be independent.
In fact, the gitea rootless container doesn't ship ssh-keygen and can be
configured to use the host's SSH server - which will cause the public
key parsing mechanism to break.

This commit changes the decision to be based on SSH.KeygenPath instead.
Any existing configurations with a custom KeygenPath set will continue
to function. The new default value of '' selects the native version. The
downside of this approach is that anyone who has relying on plain
'ssh-keygen' to have special properties will now be using the native
version instead.
I assume the exec-variant is only there because /x/crypto/ssh didn't
support ssh-ed25519 until 2016. I don't see any other reason for using
it so it might be an acceptable risk.

Fixes #23363

EDIT: this message was garbled when I tried to get the commit
description back in.. Trying to reconstruct it:

## ⚠️ BREAKING ⚠️ Users who don't have SSH.KeygenPath
explicitly set and rely on the ssh-keygen binary need to set
SSH.KeygenPath to 'ssh-keygen' in order to be able to continue using it
for public key parsing.

There was something else but I can't remember at the moment.

EDIT2: It was about `make test` and `make lint`. Can't get them to run.
To reproduce the issue, I installed `golang` in `docker.io/node:16` and
got:
```
...
go: mvdan.cc/xurls/v2@v2.4.0: unknown revision mvdan.cc/xurls/v2.4.0
go: gotest.tools/v3@v3.4.0: unknown revision gotest.tools/v3.4.0
...
go: gotest.tools/v3@v3.0.3: unknown revision gotest.tools/v3.0.3
...
go: error loading module requirements
```

Signed-off-by: Leon M. Busch-George <leon@georgemail.eu>
2023-04-11 14:34:28 +08:00
Dmitry Sharshakov eb397c3e63
Allow adding SSH keys even if SSH server is disabled (#24025)
This is useful in case you want to use them for signature verification.

A notice is added to avoid confusion.
2023-04-10 13:22:51 -04:00
Lunny Xiao c53ad052d8
Refactor the setting to make unit test easier (#22405)
Some bugs caused by less unit tests in fundamental packages. This PR
refactor `setting` package so that create a unit test will be easier
than before.

- All `LoadFromXXX` files has been splited as two functions, one is
`InitProviderFromXXX` and `LoadCommonSettings`. The first functions will
only include the code to create or new a ini file. The second function
will load common settings.
- It also renames all functions in setting from `newXXXService` to
`loadXXXSetting` or `loadXXXFrom` to make the function name less
confusing.
- Move `XORMLog` to `SQLLog` because it's a better name for that.

Maybe we should finally move these `loadXXXSetting` into the `XXXInit`
function? Any idea?

---------

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: delvh <dev.lh@web.de>
2023-02-20 00:12:01 +08:00
zeripath 52dd383b6d
Increase Content field size of gpg_key_import to MEDIUMTEXT (#22897)
Unfortunately #20896 does not completely prevent Data too long issues
and GPGKeyImport needs to be increased too.

Fix #22896

Signed-off-by: Andrew Thornton <art27@cantab.net>
2023-02-16 12:08:40 -06:00
Lunny Xiao bd820aa9c5
Add context cache as a request level cache (#22294)
To avoid duplicated load of the same data in an HTTP request, we can set
a context cache to do that. i.e. Some pages may load a user from a
database with the same id in different areas on the same page. But the
code is hidden in two different deep logic. How should we share the
user? As a result of this PR, now if both entry functions accept
`context.Context` as the first parameter and we just need to refactor
`GetUserByID` to reuse the user from the context cache. Then it will not
be loaded twice on an HTTP request.

But of course, sometimes we would like to reload an object from the
database, that's why `RemoveContextData` is also exposed.

The core context cache is here. It defines a new context
```go
type cacheContext struct {
	ctx  context.Context
	data map[any]map[any]any
        lock sync.RWMutex
}

var cacheContextKey = struct{}{}

func WithCacheContext(ctx context.Context) context.Context {
	return context.WithValue(ctx, cacheContextKey, &cacheContext{
		ctx:  ctx,
		data: make(map[any]map[any]any),
	})
}
```

Then you can use the below 4 methods to read/write/del the data within
the same context.

```go
func GetContextData(ctx context.Context, tp, key any) any
func SetContextData(ctx context.Context, tp, key, value any)
func RemoveContextData(ctx context.Context, tp, key any)
func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error)
```

Then let's take a look at how `system.GetString` implement it.

```go
func GetSetting(ctx context.Context, key string) (string, error) {
	return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) {
		return cache.GetString(genSettingCacheKey(key), func() (string, error) {
			res, err := GetSettingNoCache(ctx, key)
			if err != nil {
				return "", err
			}
			return res.SettingValue, nil
		})
	})
}
```

First, it will check if context data include the setting object with the
key. If not, it will query from the global cache which may be memory or
a Redis cache. If not, it will get the object from the database. In the
end, if the object gets from the global cache or database, it will be
set into the context cache.

An object stored in the context cache will only be destroyed after the
context disappeared.
2023-02-15 21:37:34 +08:00
zeripath 2914c5299b
Improve error report when user passes a private key (#22726)
The error reported when a user passes a private ssh key as their ssh
public key is not very nice.

This PR improves this slightly.

Ref #22693

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
2023-02-02 18:25:54 +00:00
Jason Song 477a1cc40e
Improve utils of slices (#22379)
- Move the file `compare.go` and `slice.go` to `slice.go`.
- Fix `ExistsInSlice`, it's buggy
  - It uses `sort.Search`, so it assumes that the input slice is sorted.
- It passes `func(i int) bool { return slice[i] == target })` to
`sort.Search`, that's incorrect, check the doc of `sort.Search`.
- Conbine `IsInt64InSlice(int64, []int64)` and `ExistsInSlice(string,
[]string)` to `SliceContains[T]([]T, T)`.
- Conbine `IsSliceInt64Eq([]int64, []int64)` and `IsEqualSlice([]string,
[]string)` to `SliceSortedEqual[T]([]T, T)`.
- Add `SliceEqual[T]([]T, T)` as a distinction from
`SliceSortedEqual[T]([]T, T)`.
- Redesign `RemoveIDFromList([]int64, int64) ([]int64, bool)` to
`SliceRemoveAll[T]([]T, T) []T`.
- Add `SliceContainsFunc[T]([]T, func(T) bool)` and
`SliceRemoveAllFunc[T]([]T, func(T) bool)` for general use.
- Add comments to explain why not `golang.org/x/exp/slices`.
- Add unit tests.
2023-01-11 13:31:16 +08:00
KN4CK3R 3fef47b41c
Use ErrInvalidArgument in packages (#22268)
Related to
https://github.com/go-gitea/gitea/pull/22262#discussion_r1059010774

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-12-31 12:49:37 +01:00
Gusted b76970f2e4
Fix key signature error page (#22229)
- When the GPG key contains an error, such as an invalid signature or an
email address that does not match the user.A page will be shown that
says you must provide a signature for the token.
- This page had two errors: one had the wrong translation key and the
other tried to use an undefined variable
[`.PaddedKeyID`](e81ccc406b/models/asymkey/gpg_key.go (L65-L72)),
which is a function implemented on the `GPGKey` struct, given that we
don't have that, we use
[`KeyID`](e81ccc406b/routers/web/user/setting/keys.go (L102))
which is [the fingerprint of the
publickey](https://pkg.go.dev/golang.org/x/crypto/openpgp/packet#PublicKey.KeyIdString)
and is a valid way for opengpg to refer to a key.

Before:

![image](https://user-images.githubusercontent.com/25481501/209404800-0e7c39ce-861a-455b-b234-62498d750aa8.png)

After:

![image](https://user-images.githubusercontent.com/25481501/209404821-c70f81c6-fd10-4197-ab58-61cb9fc873d8.png)

Co-authored-by: zeripath <art27@cantab.net>
2022-12-30 12:53:05 +08:00
Lunny Xiao 0a7d3ff786
refactor some functions to support ctx as first parameter (#21878)
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-12-03 10:48:26 +08:00
flynnnnnnnnnn e81ccc406b
Implement FSFE REUSE for golang files (#21840)
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2022-11-27 18:20:29 +00:00
Lunny Xiao 34283a74e8
Allow detect whether it's in a database transaction for a context.Context (#21756)
Fix #19513

This PR introduce a new db method `InTransaction(context.Context)`,
and also builtin check on `db.TxContext` and `db.WithTx`.
There is also a new method `db.AutoTx` has been introduced but could be used by other PRs.

`WithTx` will always open a new transaction, if a transaction exist in context, return an error.
`AutoTx` will try to open a new transaction if no transaction exist in context.
That means it will always enter a transaction if there is no error.

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: 6543 <6543@obermui.de>
2022-11-12 21:18:50 +01:00
delvh 0ebb45cfe7
Replace all instances of fmt.Errorf(%v) with fmt.Errorf(%w) (#21551)
Found using
`find . -type f -name '*.go' -print -exec vim {} -c
':%s/fmt\.Errorf(\(.*\)%v\(.*\)err/fmt.Errorf(\1%w\2err/g' -c ':wq' \;`

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-10-24 20:29:17 +01:00
zeripath 716fcfcf72
Make every not exist error unwrappable to a fs.ErrNotExist (#20891)
A lot of our code is repeatedly testing if individual errors are
specific types of Not Exist errors. This is repetitative and unnecesary.
`Unwrap() error` provides a common way of labelling an error as a
NotExist error and we can/should use this.

This PR has chosen to use the common `io/fs` errors e.g.
`fs.ErrNotExist` for our errors. This is in some ways not completely
correct as these are not filesystem errors but it seems like a
reasonable thing to do and would allow us to simplify a lot of our code
to `errors.Is(err, fs.ErrNotExist)` instead of
`package.IsErr...NotExist(err)`

I am open to suggestions to use a different base error - perhaps
`models/db.ErrNotExist` if that would be felt to be better.


Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
2022-10-18 07:50:37 +02:00
zeripath 9d6a2034bc
Increase Content field size of gpg_key and public_key to MEDIUMTEXT (#20896)
Unfortunately some keys are too big to fix within the 65535 limit of TEXT on MySQL
this causes issues with these large keys.

Therefore increase these fields to MEDIUMTEXT.

Fix #20894

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-08-22 14:32:28 +01:00
zeripath 11bae50484
Pad GPG Key ID with preceding zeroes (#20878) 2022-08-21 02:50:15 -04:00
Lunny Xiao 86c85c19b6
Refactor AssertExistsAndLoadBean to use generics (#20797)
* Refactor AssertExistsAndLoadBean to use generics

* Fix tests

Co-authored-by: zeripath <art27@cantab.net>
2022-08-16 10:22:25 +08:00
Wim cb50375e2b
Add more linters to improve code readability (#19989)
Add nakedret, unconvert, wastedassign, stylecheck and nolintlint linters to improve code readability

- nakedret - https://github.com/alexkohler/nakedret - nakedret is a Go static analysis tool to find naked returns in functions greater than a specified function length.
- unconvert - https://github.com/mdempsky/unconvert - Remove unnecessary type conversions
- wastedassign - https://github.com/sanposhiho/wastedassign -  wastedassign finds wasted assignment statements.
- notlintlint -  Reports ill-formed or insufficient nolint directives
- stylecheck - https://staticcheck.io/docs/checks/#ST - keep style consistent
  - excluded: [ST1003 - Poorly chosen identifier](https://staticcheck.io/docs/checks/#ST1003) and [ST1005 - Incorrectly formatted error string](https://staticcheck.io/docs/checks/#ST1005)
2022-06-20 12:02:49 +02:00
Lunny Xiao 26095115f4
Move some repository related code into sub package (#19711)
* Move some repository related code into sub package

* Move more repository functions out of models

* Fix lint

* Some performance optimization for webhooks and others

* some refactors

* Fix lint

* Fix

* Update modules/repository/delete.go

Co-authored-by: delvh <dev.lh@web.de>

* Fix test

* Merge

* Fix test

* Fix test

* Fix test

* Fix test

Co-authored-by: delvh <dev.lh@web.de>
2022-06-06 16:01:49 +08:00
Lunny Xiao 12c742f8dc
Fix order by parameter (#19849)
Upgrade builder to v0.3.11
Upgrade xorm to v1.3.1 and fixed some hidden bugs.

Replace #19821
Replace #19834
Included #19850

Co-authored-by: zeripath <art27@cantab.net>
2022-06-04 20:18:50 +01:00
Lunny Xiao fd7d83ace6
Move almost all functions' parameter db.Engine to context.Context (#19748)
* Move almost all functions' parameter db.Engine to context.Context
* remove some unnecessary wrap functions
2022-05-20 22:08:52 +08:00
Lunny Xiao b8911fb456
Use a struct as test options (#19393)
* Use a struct as test options

* Fix name

* Fix test
2022-04-14 21:58:21 +08:00
wxiaoguang 2b55422cd7
Fix the bug: deploy key with write access can not push (#19010)
Use DeployKeyID to replace the IsDeployKey, then CanWriteCode uses the DeployKeyID to check the write permission.
2022-03-22 17:29:07 +08:00
zeripath 76e3111596
Collaborator trust model should trust collaborators (#18539)
* Collaborator trust model should trust collaborators

There was an unintended regression in #17917 which leads to only
repository admin commits being trusted. This PR restores the old logic.

Fix #18501

Signed-off-by: Andrew Thornton <art27@cantab.net>
2022-02-02 11:10:06 +01:00
luzpaz 8c647bf0f6
Fix various typos (#18219)
Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby,te,unknwon`

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-01-10 17:32:37 +08:00
Gusted 242dddfcb7
Remove ioutil (#18222)
- Don't use `ioutil` package anymore as it doesn't anything special
anymore since Go 1.16:

```
// As of Go 1.16, the same functionality is now provided
// by package io or package os, and those implementations
// should be preferred in new code.
```

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-01-10 09:48:13 +08:00
Lunny Xiao de8e3948a5
Refactor auth package (#17962) 2022-01-02 21:12:35 +08:00
6543 660c30db80
Migration 204 use Sync2 (#18044)
* format nit

* Sync2
2021-12-20 18:58:38 +01:00
Wim 6fe756dc93
Add support for ssh commit signing (#17743)
* Add support for ssh commit signing

* Split out ssh verification to separate file

* Show ssh key fingerprint on commit page

* Update sshsig lib

* Make sure we verify against correct namespace

* Add ssh public key verification via ssh signatures

When adding a public ssh key also validate that this user actually
owns the key by signing a token with the private key.

* Remove some gpg references and make verify key optional

* Fix spaces indentation

* Update options/locale/locale_en-US.ini

Co-authored-by: Gusted <williamzijl7@hotmail.com>

* Update templates/user/settings/keys_ssh.tmpl

Co-authored-by: Gusted <williamzijl7@hotmail.com>

* Update options/locale/locale_en-US.ini

Co-authored-by: Gusted <williamzijl7@hotmail.com>

* Update options/locale/locale_en-US.ini

Co-authored-by: Gusted <williamzijl7@hotmail.com>

* Update models/ssh_key_commit_verification.go

Co-authored-by: Gusted <williamzijl7@hotmail.com>

* Reword ssh/gpg_key_success message

* Change Badsignature to NoKeyFound

* Add sign/verify tests

* Fix upstream api changes to user_model User

* Match exact on SSH signature

* Fix code review remarks

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-12-19 00:37:18 -05:00
Lunny Xiao 3ca5dc7e32
Move keys to models/asymkey (#17917)
* Move keys to models/keys

* Rename models/keys -> models/asymkey

* change the missed package name

* Fix package alias

* Fix test

* Fix docs

* Fix test

* Fix test

* merge
2021-12-10 16:14:24 +08:00