Commit Graph

1631 Commits

Author SHA1 Message Date
Olivier Goffart
a902726938 Restoration items should appear in the sync protocol
When an item is downloaded because it is restored, it shall be shown in the
sync protocol.
(It is also going to be shown in the not synchronized for a short while, but
that's fine)
2019-10-11 13:09:07 +02:00
Olivier Goffart
06e3a98e8d Fix Upload of large (> 2GiB) files
Issue #7506

This is a regression introduced by the delta sync feature (as the chunk offset
changed from being the chunk number to be the byte offset, it needs to be a
qint64 now)
2019-10-09 13:55:06 +02:00
Christian Kamm
ca3819eefe Discovery: Distinguish readdir and closedir errors 2019-10-08 09:37:29 +02:00
Olivier Goffart
cc1cb8c78e Fix warning about serverJob not being used
And fix a FIXME in the same time
2019-10-07 14:50:44 +02:00
Christian Kamm
5f3682a47d Propagate dir: Never write the etag on remote mkdir #7481
It must always only be written once all children are successfully
propagated.
2019-10-07 14:33:15 +02:00
Christian Kamm
976db2ef84 Test: Disable local discovery parallelism in permission test
Adding parallelism broke the test because it depended on the order of
discovery.
2019-09-16 19:52:19 +02:00
Hannah von Reth
ed73139e9f Fix build with old gcc
Old gcc doesn't allow aggregate initialisation of strcuts initialised in class

Fixes: #7451
2019-09-16 14:56:07 +02:00
Christian Kamm
80e812399a Discovery: Change local job signal signatures
To make it more explicit that data a copy of the data is transfered
between threads. (though Qt will make a copy of the arguments anyway)
2019-09-11 15:50:37 +02:00
Christian Kamm
3218df49da Discovery: local job shouldn't be parented
Since it'll be deleted by the thread pool.
2019-09-11 15:50:37 +02:00
Christian Kamm
fa49b3d522 Don't store pointer to local job
There were crashes in the QPointer assignment. Possibly the thread pool
is done with the job and deletes it before the assignment is done.
2019-09-11 11:31:30 +02:00
Markus Goetz
e1b31d742c Discovery: List local directories from thread #7456 #7439 2019-09-09 20:41:55 +02:00
Markus Goetz
e81400c090 Reconcile: Sort already during discovery #7445 2019-09-09 11:39:44 +02:00
Markus Goetz
85aad7629d Propagator: Delay job execution a bit #7439 2019-09-02 10:27:31 +02:00
Christian Kamm
d3df7532db Discovery: Remove level of indent around moves
No code changes.
2019-09-02 09:58:59 +02:00
Christian Kamm
99fd45051a Discovery: If a move is forbidden, restore the source
Previously the source was deleted (or attempted to be deleted), even if
the new location was not acceptable for upload. This could make data
unavilable on the server.

For #7410
2019-09-02 09:58:59 +02:00
Christian Kamm
1e652e12b5 Propagation: Fix delete-before-rename bug #7441
By introducing a PropagateRootDirectory job that explicitly
separates the directory deletion jobs from all the other jobs.

Note that this means that if there are errors in subJobs the
dirDeletionJobs won't get executed.
2019-08-30 18:33:23 +02:00
Markus Goetz
30e924ed8d Propagator: Make sure we schedule only one job #7439
To not starve the event loop.
(There is still ~= 3 jobs running at the same time)
2019-08-30 15:46:39 +02:00
Christian Kamm
2820ac14f4 PropagateDownload: Don't try to open readonly temporaries
This situation could arrise when receiving a read-only share and the
temporary rename failed for some reason.

See #7419
2019-08-26 07:48:09 +02:00
Christian Kamm
7da58fcf27 Don't fatal on "Storage temporarily unavailable"
This is an unreliable workaround. The real fix will need to be deferred
to another release.

For #5088
2019-08-22 13:05:01 +02:00
Hannah von Reth
c4f83fe382 CMake VFS: Enable us to provided vfs plugins from an external directory 2019-08-19 15:37:05 +02:00
Hannah von Reth
ed0f10f2b6 Print critical and fatal messages to stderr
That way we might get a reason why the client crashes (dev setup)
2019-08-14 09:07:03 +02:00
Christian Kamm
98c04e1b92 Remove maxLogLines config option
It's no longer used.

For owncloud/docs#1365
2019-08-13 13:24:16 +02:00
Christian Kamm
224129957b Download: Don't trigger too many concurrent hash computations
Previously the job would only become "active" when the downloads
started. That meant that arbitrarily many hash computations could be
queued at the same time.

Since the the file was opened during future creation this could lead to
a "too many open files" problem if there were lots of new-new conflicts.

To change this:
- Make PropagateDownload become active when computing a hash
  asynchronously.
- Make the computation future open the file only once it gets run. This
  will make it less likely for this problem to occur even if thousands
  of these futures are queued.

For #7372
2019-08-12 12:27:59 +02:00
Christian Kamm
b7ab14db72 Fix 'unused parameter name' warnings 2019-08-12 08:08:26 +02:00
Christian Kamm
907945d869 LogDir: Compress last logfile on restart
For #7353
2019-08-01 10:12:50 +02:00
Christian Kamm
b3a3f17d74 SyncEngine: Don't duplicate fatal errors
Previously fatal error texts were duplicated: Once they entered the
SyncResult via the SyncFileItem and once via syncError().

syncError is intended for folder-wide sync issues that are not pinned
to particular files. Thus that duplicated path is removed.

For #5088
2019-07-31 09:02:02 +02:00
Christian Kamm
23f005e60d Discovery: Parse etag to be consistent with RequestEtagJob
This avoids unnecessary sync runs.

For #7345
2019-07-31 09:01:41 +02:00
Christian Kamm
4fd9bdc4b4 Vfs: Remove old db record when dehydrating via rename
For #7338
2019-07-25 15:21:54 +02:00
Christian Kamm
db346dfc08 Vfs: Make move detection work with virtual files #7001
Previously a checksum computation could be done on a suffix-placeholder
file, making discovery believe that no move took place.
2019-07-24 16:16:55 +02:00
Christian Kamm
d09c4fd2e8 Vfs: Ensure pins change with (de-)hydration
Previously an implicit hydration of a file in an online-only folder
would not change the pin state and cause a dehydration on the next
sync.
2019-07-24 16:16:39 +02:00
Christian Kamm
c44ca94e07 Vfs: Add hook to allow update-metadata for unchanged files
Allows winvfs to convert files to placeholders when vfs is enabled. This
is required to mark files as in-sync #7329.
2019-07-24 10:50:39 +02:00
Christian Kamm
199163187a Vfs: Preserve pin state on hydration
For #7322 and #7323
2019-07-18 20:00:28 +02:00
Dominik Schmidt
3e4b3928c2 Add missing OWNCLOUDSYNC_EXPORT 2019-06-28 17:27:33 +02:00
Christian Kamm
6b89747a26 VfsSuffix: Wipe stale pin states #7273
Previously the pin states of deleted files stayed in the 'flags'
database and could be inadvertently reused when a new file with the same
name appeared. Now they are deleted.

To make this work right, the meaning of the 'path' column in the 'flags'
table was changed: Previously it never had the .owncloud file suffix.
Now it's the same as in metadata.path.

This takes the safe parts from #7274 for inclusion in 2.6. The more
elaborate database schema changes (why use 'path' the join the two
tables in the first place?) shall go into master.
2019-06-27 15:57:20 +02:00
Christian Kamm
86b15ffb20 RequestEtagJob: Consistently parse etags #7271
Previously RequestEtagJob did return the etag verbatim (including extra
quotes) while the db had the parsed form. That caused the etag
comparison during discovery move detection to always fail. The test
didn't catch it because the etags there didn't have quotes.

Now:
- RequestEtagJob will parse the etag, leading to a consistent format
- Tests have etags with quotes, detecting the problem
2019-06-27 13:08:18 +02:00
Christian Kamm
b99c08904e UploadDevice: Fix windows issues #7264
- Close the UploadDevice to close the QFile after the PUT job is done.
  This allows winvfs to get an oplock on the file later.

- Don't rely on QFile::fileName() to be valid after
  openAndSeekFileSharedRead() was called. The way it is openend on
  Windows makes it have an empty filename.
2019-06-23 10:31:50 +02:00
Christian Kamm
02a9b16b0f Vfs: Move pin state if files move #7250
Previously renames of items didn't carry the pin state with them.
2019-06-23 10:23:37 +02:00
Christian Kamm
e653924e06 Vfs: Don't let new local files start out unpinned #7250
If one adds a new file to an online-only folder the previous behavior
was to upload the file in one sync and dehydrate it in the next. Now
these new files get set to Unspecified pin state, making them retain
their data.
2019-06-23 10:23:37 +02:00
Christian Kamm
561460bff7 Shell extension: Show "show versions" only when available #7196
- Don't show it for directories.
- Don't show it when versioning is disabled.
2019-06-16 12:09:52 +02:00
Christian Kamm
86d36f0a78 Merge remote-tracking branch 'origin/2.5' into 2.6 2019-06-12 13:23:41 +02:00
Christian Kamm
98ba64e60c Http2: Resend requests on ContentReSend error #7174
Since Qt does not yet transparently resend HTTP2 requests in some cases
we do it manually.

The test showed a problem where the initial non-200 reply would close
the target temporary file and the follow-up request couldn't store any
data. Removing that close() call is safe because there also is a
_saveBodyToFile flag that guards writes to the target file.

(cherry picked from commit 677e44dc53)

Cherry-picked to allow 2.5 users with new enough Qt to test the HTTP2
workaround.
2019-06-12 13:16:32 +02:00
Christian Kamm
42e4d1061f Upload: Read file chunks gradually #7226
Instead of all at once, to reduce peak memory use.

Changing UploadDevice in this way requires keeping the file open for the
duration of the upload. It also means changes to open(), seek(), close()
to ensure that uses of the device work right when a request needs to
be resent.
2019-06-12 13:09:50 +02:00
Christian Kamm
fd523c3b0f HTTP2: Enable if using Qt >=5.12.4 #7092
This Qt version fixes QTBUG-73947 and along with the resend workaround
for #7174 HTTP2 should work more reliably.
2019-06-07 19:26:44 +02:00
Christian Kamm
677e44dc53 Http2: Resend requests on ContentReSend error #7174
Since Qt does not yet transparently resend HTTP2 requests in some cases
we do it manually.

The test showed a problem where the initial non-200 reply would close
the target temporary file and the follow-up request couldn't store any
data. Removing that close() call is safe because there also is a
_saveBodyToFile flag that guards writes to the target file.
2019-06-07 19:26:19 +02:00
Christian Kamm
d7d55865fb SyncEngine: Don't close db when done #7141
The db-close operation is likely a leftover from when the SyncEngine
owned its own db connection and serves no purpose anymore.

Closing the db causes the removal of the temporary wal and shm files.
These files are recreated when the db is opened again, which happens
almost immediately.

This is a problem for winvfs because the delete-recreate step wipes the
exclusion state on these files just after the sync is done. That meant
that the db temporaries permanently had a "needs sync" icon marker shown
in the explorer.

Avoiding reopening the db also reduces the number of log messages per
sync.

(cherry picked from commit 2a900901d3)

Cherry picked as it seems likely to decrease the severety of #6877: if
the database isn't closed and reopened as frequently users are less
likely to run into crashes when the sqlite connection switches to wal
journaling mode.
2019-06-07 09:30:52 +02:00
Olivier Goffart
cb26f516f7 Discovery: Do not abort the sync in case of error 404 (or 500)
Issue: #7199
2019-06-05 11:09:16 +02:00
Olivier Goffart
819175ad89 SyncEngine: Fix renaming a single file cause the "delete all file" popup
Possibly a regression, since the new discovery discovers rist the renamed
files as removed

Issue #7204
2019-06-04 15:56:35 +02:00
Christian Kamm
dbae6bf224 SocketAPI: Add "show versions" to shell context menu #7196
The new menu entry depends on the privateLinksDetailsParam
capability of the server.
2019-06-04 12:26:15 +02:00
Markus Goetz
dc2a37e813 OAuth2: Better error logging
This does not fix a bug, just was found while spotting a bug that was no bug.
For https://github.com/owncloud/enterprise/issues/2951
2019-05-15 09:58:59 +02:00
Christian Kamm
87cd1c38be Merge remote-tracking branch 'origin/2.5' into 2.6 2019-05-13 07:51:54 +02:00