Commit Graph

1769 Commits

Author SHA1 Message Date
Kevin Ottens
cfd3771818
Allow to override the parallelism
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-15 13:05:11 +02:00
Kevin Ottens
5cfbf34501
Move PropagateRemoteMkdir ctor to the cpp file
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-15 13:03:55 +02:00
Dominique Fuchs
19d46f0541 Merge branch 'fix-delete-base-folders-instr' of https://github.com/Nextcloud/desktop into fix-delete-base-folders-instr 2020-07-09 21:47:34 +02:00
Dominique Fuchs
f8920f969f Remove debug variable
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2020-07-09 21:47:19 +02:00
Dominique Fuchs
8deb38ac24
Merge branch 'master' into fix-delete-base-folders-instr 2020-07-09 21:38:11 +02:00
Dominique Fuchs
d47e570ff3 Prevent empty FileMap in local csync to be processed if folder not empty
Signed-off-by: Dominique Fuchs <32204802+DominiqueFuchs@users.noreply.github.com>
2020-07-09 21:23:24 +02:00
Kevin Ottens
3ccac1b1be Properly register folder as encrypted during sync
Turns out that when we added a new e2e folder during sync, we were
passing the wrong path to the e2e object. We have several path
convention in the sync code, just happened to be the wrong one.

I still long for the day when we'll use the type system to deal with
paths. All those strings are error-prone.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-09 17:16:21 +02:00
Kevin Ottens
26686ecd45 Recover from missing metadata on uploads
Other clients seem to allow creating empty encrypted folders with no
metadata associated to them, so let's get ready to handle it. In case of
a 404 to get the metadata, we create an empty metadata payload and pass
it further down.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-09 16:22:54 +02:00
Kevin Ottens
4ef42e53d3 Change the separator in the private key
It used to be a base64 encoded '|', now it is still a '|' but not
encoded, let's adjust accordingly.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-09 16:22:54 +02:00
Kevin Ottens
3c20b2414e Prevent parallelism when deleting encrypted files
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-09 16:22:54 +02:00
Kevin Ottens
1be869fdc7 Unlock the folder only after we actually deleted the file
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-09 16:22:54 +02:00
Kevin Ottens
8e0ab5ce47 Allow to specify the e2e-token on DeleteJob
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-09 16:22:54 +02:00
Kevin Ottens
0575d0e793 Stop unlocking from within PropagateRemoteDeleteEncrypted
We need to finish deleting the file before we can actually unlock. Also
the token will be necessary for the delete to succeed.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-09 16:22:54 +02:00
Kevin Ottens
7e251edd6e Compare the unencrypted filename with the originalName
Otherwise obviously we'd never find the files in the metadata.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-09 16:22:54 +02:00
Kevin Ottens
2831b22be4 Use the encrypted path for collection listing
If we use the local path we'll get a 404.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-09 16:22:54 +02:00
Kevin Ottens
511a6e66dc Pass e2e-token to PropagateUpload subjobs
The headers() method is used to pass extra headers to the PUT jobs for
instance, definitely needed for uploads now.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-09 16:22:54 +02:00
Kevin Ottens
9a4c63404d Pass the e2e-token for encrypted MKCOL
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-09 16:22:54 +02:00
Kevin Ottens
01f5501ad8 Allow to pass extraHeaders to a path based MkColJob
We could specify the extra headers with a URL but not a simple path.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-09 16:22:54 +02:00
Kevin Ottens
e14ae806b6 E2EE RFC has been updated to use e2e-token instead of token
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-09 16:22:54 +02:00
Stephan Beyer
293be2c2e6 Remove redundant initialization of HttpCredentials::_keychainMigration
The value gets already initialized by default member initialization,
so there is no need to set it again.

Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
2020-07-08 14:08:41 +02:00
Stephan Beyer
a9de465927 Remove redundant initialization of SyncFileStatus::_shared attribute
The value gets already initialized by default member initialization,
so there is no need to set it again in the copy constructor.

Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
2020-07-08 14:08:41 +02:00
Stephan Beyer
ed570d6274 Use default member init for PKeyCtx::_ctx and PKey::_pkey
Signed-off-by: Stephan Beyer <s-beyer@gmx.net>
2020-07-08 14:08:41 +02:00
Kevin Ottens
c2632eb604 Handle properly null avatar
If we try to produce the avatar before we managed to connect we'd end up
trying to paint in a null image. Just return early, which will allow the
caller to do something else instead.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-07 18:07:33 +02:00
Kevin Ottens
217869e95c Decode the href from the XML payload
The XML we get as reply has href entities properly percent encoded.
Since we didn't go through QUrl next we didn't get a properly decoded
version. Make sure we decode it before storage.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-07 15:51:24 +00:00
Kevin Ottens
f90d6951c2 No need to compute base at each loop run
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-07 15:51:24 +00:00
Michael Schuster
f4d83d02f6
Cleanup auto pointers and qobject casts, refactor KeychainChunk
Signed-off-by: Michael Schuster <michael@schuster.ms>
2020-07-06 21:51:40 +02:00
Michael Schuster
8503226c44
Keychain: Use auto deletion in WebFlowCredentials and ConfigFile
- Also make use of the new KeychainChunk::DeleteJob

Signed-off-by: Michael Schuster <michael@schuster.ms>
2020-07-06 21:51:40 +02:00
Michael Schuster
42eb3388f8
Refactor KeychainChunk to use QEventLoop and add DeleteJob class
- Use QEventLoop for synchronous exec()
- Rename startAwait() to exec()
- Add code for auto deletion
- Add new DeleteJob class
- Cleanup, tweaks

Signed-off-by: Michael Schuster <michael@schuster.ms>
2020-07-06 21:51:40 +02:00
Michael Schuster
81c644e702
ConfigFile security: Migrate Proxy password to keychain
When specified in the config file, the Proxy password will be migrated
to the keychain, for backward compatibility and to allow admins to
overwrite an existing password by rolling out updated config files.

Once migrated to the keychain, the password will be removed from the
config file.

Signed-off-by: Michael Schuster <michael@schuster.ms>
2020-07-06 21:51:39 +02:00
Michael Schuster
18cbbc5052
KeychainChunk: Add synchronous method startAwait()
Awaits completion with no need to connect some slot to the finished() signal first,
inspired by the ProxyAuthHandler class.

Also add:
- Job dtor to safely erase passwords
- textData() method
- New ctor overloads to work with arbitrary keys (without Account ptrs)

Signed-off-by: Michael Schuster <michael@schuster.ms>
2020-07-06 21:51:39 +02:00
Michael Schuster
2a3ef044be
Move KeychainChunk class from gui to libsync
Signed-off-by: Michael Schuster <michael@schuster.ms>
2020-07-06 21:51:39 +02:00
asemmar
628a426d22 forgot some indent
Signed-off-by: asemmar <anis.semmar@itron.com>
2020-07-06 09:46:14 +02:00
asemmar
d5587e386c fixed bad spelling of word 'retreive' except in sqlite dependancy
Signed-off-by: asemmar <anis.semmar@itron.com>
2020-07-06 09:46:14 +02:00
Kevin Ottens
0b1cf69134 Update the metadata table with encryption info before discovery
When the ClientSideEncryption object is fed, also serialize the
encryption info of the folders inside the metadata table.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-06 05:27:14 +00:00
Kevin Ottens
287a130044 Also deliver the encrypted status via the ClientEncryption signal
This will turn useful for other consumers of that data. The alternative
would be to expose a method breaking all form of encapsulation.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-06 05:27:14 +00:00
Kevin Ottens
3520360159 Move the e2e information fetch even earlier
We used to do it when the propagation starts, let's do it even before
the discovery starts. This way we'll have a chance to exploit the
information during the discovery phase.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-06 05:27:14 +00:00
Kevin Ottens
f98c7990ba Split the discovery phase out of startSync()
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-06 05:27:14 +00:00
Kevin Ottens
5a07a36d06 Avoid job parallelism when uploading to an encrypted folder
With the current design of the file upload this necessarily pushed to a
lock starvation on the folder. Indeed you could end up with N jobs
asking for the lock at the same time. So just avoid parallelizing for
now even though it will be slow.

We could try to optimize but that'd require some serious changes to the
sync logic on the jobs... let's stabilize first and optimize later.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-01 17:11:57 +00:00
Kevin Ottens
27b982ceaf Override parallelism() for uploads
This way we'll be able to control parallelism for upload jobs.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-01 17:11:57 +00:00
Kevin Ottens
5e20e2b46f Move PropagateUpload ctor impl to the cpp file
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-01 17:11:57 +00:00
Kevin Ottens
d2e8cd4469 Deal properly with encrypted item updates accross connections
In such cases we get a download for which _file is already the demangled
name and _encryptedFileName has the mangled information. This is
different to what we encountered so far where initially _file was
mangled and _encryptedFileName was empty. Let's deal with that case
properly.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-01 16:58:29 +00:00
Kevin Ottens
3204c15911 Better separate between absolute and relative paths on uploads
Yes... I still wish this would be all driven by the type system, would be
much less error-prone.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-01 16:58:29 +00:00
Kevin Ottens
8d5c79c219 Better separate between absolute and relative paths on downloads
I wish this would be all driven by the type system instead of
error-prone string concatenation everywhere. That will be for a (much)
later refactoring hopefully.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-01 16:58:29 +00:00
Kevin Ottens
8a181adb3a Always use an absolute remote path for GetFolderEncryptStatusJob
It turns out this job expected an absolute remote path even in the case
of a subfolder sync point.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-01 16:58:29 +00:00
Kevin Ottens
fdceb1f2d2 Allow to control logFlush from the config file
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-01 16:10:56 +02:00
Kevin Ottens
f557dd7792 Allow to control logDir from the config file
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-01 16:10:53 +02:00
Kevin Ottens
be2d9d4838 Allow to control logExpire from the config file
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-01 16:10:50 +02:00
Kevin Ottens
808fb17809 Allow to control log debug from the config file
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-01 15:56:23 +02:00
Kevin Ottens
7996eaae82 Have a filter rule capturing all of nextcloud debug
This was half broken before that commit and the previous one since some
of the categories would not be captured.

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-01 15:39:52 +02:00
Kevin Ottens
53c31f5fed Make sure all our logging categories start with nextcloud.
Otherwise it was a bit confusing and annoying for filter rules:
e.g. "nextcloud.sync.*" vs "sync.*".

Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
2020-07-01 15:39:52 +02:00