Thanks to the new discovery algorithm we got both mangled and original
file names in the item so no need to go through the database for
nothing.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
This was done because the propagator jobs where running in a thread a long
time ago, but this is no longer the case.
(Also QAtomicInt::load is marked as deprecated now)
Previously the job would only become "active" when the downloads
started. That meant that arbitrarily many hash computations could be
queued at the same time.
Since the the file was opened during future creation this could lead to
a "too many open files" problem if there were lots of new-new conflicts.
To change this:
- Make PropagateDownload become active when computing a hash
asynchronously.
- Make the computation future open the file only once it gets run. This
will make it less likely for this problem to occur even if thousands
of these futures are queued.
For #7372
Previously the pin states of deleted files stayed in the 'flags'
database and could be inadvertently reused when a new file with the same
name appeared. Now they are deleted.
To make this work right, the meaning of the 'path' column in the 'flags'
table was changed: Previously it never had the .owncloud file suffix.
Now it's the same as in metadata.path.
This takes the safe parts from #7274 for inclusion in 2.6. The more
elaborate database schema changes (why use 'path' the join the two
tables in the first place?) shall go into master.
Since Qt does not yet transparently resend HTTP2 requests in some cases
we do it manually.
The test showed a problem where the initial non-200 reply would close
the target temporary file and the follow-up request couldn't store any
data. Removing that close() call is safe because there also is a
_saveBodyToFile flag that guards writes to the target file.
As far as I'm aware local discovery can be skipped on folders that are
selective-sync blacklisted, so a local discovery is required when an
entry is removed from the blacklist.
Also rename
avoidReadFromDbOnNextSync() -> schedulePathForRemoteDiscovery()
since the old name might also imply it's not read from db in the local
discovery - which is not the case. Use Folder::
schedulePathForLocalDiscovery() for that.
Creating a new virtual file and replacing a file with a virtual one now
have their own text in the protocol, not just "Downloaded".
To do this, the SyncFileItem type is kept as
ItemTypeVirtualFileDehydration for these actions. Added new code to
ensure the type isn't written to the database.
While looking at this, I've also added documentation on SyncFileItem's
_file, _renameTarget, _originalFile and destination() because some of
the semantics weren't clear.
The block of code that propagated attributes etc from the previously
existing file was placed *after* the block that renamed the previously
existing file to a conflict name. That meant the propagation didn't work
in the conflict case.
Since 'placeholder' just means that it's an item of the special type
that the vfs plugin can deal with - no matter whether hydrated or
dehydrated - all done items should become placeholders. Even
directories.
Now every file that passes through updateMetadata() will be converted to
a placeholder if necessary.
OAuth2 access token typically only has a token valid for 1 hour.
Before this patch, when the token was timing out during the sync, the
sync was aborted, and the ConnectionValidator was then requesting a new
token, so the sync can be started over.
If the discovery takes longer than the oauth2 validity, this means that
the sync can never proceed, as it would be always restarted from scratch.
With this patch, we try to transparently renew the OAuth2 token and restart
the jobs that failed because the access token was invalid.
Note that some changes were required in the GETFile job because it handled
the error itself and so it was erroring the jobs before its too late.
Issue #6814
Unfortunately to do this, the local update phase must write to the
database, creating a new side-effect and order dependency (local update
must run before remote update).
We want to keep the body so we can get the message from it
(Issue #6459)
TestDownload::testErrorMessage did not fail because the FakeErrorReply
did not emit readyRead and did not implement bytesAvailable.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
Otherwise a interrupted or unsuccessful download would mean that the
download-intend was forgotten. The next sync would reestablish the
virtual file instead.
Issue #6420
Store the X-Request-ID in the SyncFileItem and also in the blacklist.
Note that for consistency reason, the X-Request-ID is also in the
SyncFileItem if the request succeeds.
Currently there is no UI to access it, but it can be queried with sql
commands
From issue #7015, the code is wrong because the path is the file system path and
not the path on the DB.
But since this is a conflict, this means the reconcile will still want to download
the file from the server next sync, so we need not to worry about this case
Now the db entries for placeholders will have the full placeholder
paths. That way older clients will, on remote discovery, delete the
placeholders and download the real files.
- Controled by an option.
- New remote files start out as ItemTypePlaceholder, are created with a
.owncloud extension.
- When their db entry is set to ItemTypePlaceholderDownload the next
sync run will download them.
- Files that aren't in the placeholder state sync as usual.
- See test cases in testsyncplaceholders.
Missing:
- User ui for triggering placeholder file download
- Maybe: Going back from file to placeholder?
In such cases we get a download for which _file is already the demangled
name and _encryptedFileName has the mangled information. This is
different to what we encountered so far where initially _file was
mangled and _encryptedFileName was empty. Let's deal with that case
properly.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
I wish this would be all driven by the type system instead of
error-prone string concatenation everywhere. That will be for a (much)
later refactoring hopefully.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
This means adjusting PropagateDownloadEncrypted so that it knows where
the file will end (otherwise it would create temporary files in non
existant paths for instance).
In turn we have to adjust PropagateDownloadFile accordingly so that it
resolves the local folder the file will end up in.
And last we adjust PropagateLocalMkdir to resolve paths as well and
demangle as needed.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
We give them a parent to make sure they will be destroyed when the jobs
which created them are destroyed themselves.
Signed-off-by: Kevin Ottens <kevin.ottens@nextcloud.com>
