Commit Graph

87 Commits

Author SHA1 Message Date
Christian Kamm
15ee7b39ac Merge remote-tracking branch 'origin/2.3' 2017-06-22 14:06:31 +02:00
Christian Kamm
4291ea47f7 Journal: Don't use a ._ path if it won't work #5633
When synchronizing a folder on a samba share, creating files that begin
with ._ is often forbidden. This prevented the client from creating
its ._sync_abcdef.db file.

Now, it'll check whether the preferred filename is creatable, and if
it isn't it'll use .sync_abcdef.db instead.

The disadvantage is that this alternative path won't be ignored by
older clients - that was the reason for the ._ prefix.
2017-06-20 13:35:34 +02:00
Christian Kamm
8160963110 Compare the hash of files with identical mtime/size #5589
* For conflicts where mtime and size are identical:

  a) If there's no remote checksum, skip (unchanged)
  b) If there's a remote checksum that's a useful hash, create a
     PropagateDownload job and compute the local hash. If the hashes
     are identical, don't download the file and just update metadata.

* Avoid exposing the existence of checksumTypeId beyond the database
  layer. This makes handling checksums easier in general because they
  can usually be treated as a single blob.

  This change was prompted by the difficulty of producing file_stat_t
  entries uniformly from PROPFINDs and the database.
2017-06-15 13:54:16 +02:00
Christian Kamm
c8d0f788e0 Apply clang-format 2017-05-17 12:26:27 +02:00
Christian Kamm
df3fe25702 Reformatting: Adjust trailing comments
These would otherwise be line-wrapped by clang-format,
and then consecutive reformattings remove the aligned
comment indentation

Example:

int a; // too long comment

->

int a; // too long
       // comment

->

int a; // too long
// comment
2017-05-17 12:26:27 +02:00
Jocelyn Turcotte
b7553d5bdf Upgrade some qCDebug to qCInfo or qCWarning
Use qCInfo for anything that has general value for support and
development. Use qCWarning for any recoverable error and qCCritical
for anything that could result in data loss or would identify a serious
issue with the code.

Issue #5647
2017-05-11 17:22:59 +02:00
Jocelyn Turcotte
4ad190a558 Use Qt logging categories for logging
This gives more insight about the logs and allow setting fine-tuned
logging rules. The categories are set to only output Info by default
so this allows us to provide more concise logging while keeping the
ability to extract more information for a specific category when
developping or debugging customer issues.

Issue #5647
2017-05-11 17:22:59 +02:00
Olivier Goffart
e8c10501a5 Folder: Keep files option after aboutToRemoveAllFiles should not clear selective sync list (#5531)
We were removing the wholme journal db when the user wanted to keep all files,
But that would also remove the selective sync lists.
We should only remove the metadata table.

Issue #5484
2017-02-15 13:27:41 +01:00
Christian Kamm
4c1fdf1dee Double check usage of asserts #5429
A few are supposed to be fatal.
2017-02-08 15:15:19 +01:00
Olivier Goffart
a63d970e5e ChunkingNG: remove stale chunks when cleaning the uploadInfo table
Stale chunks might be there because a file was removed or would just not
be uploaded, for any reason.
We just start the DeleteJob but we don't care if it success or not.

Relates to https://github.com/owncloud/core/issues/26981

One of the test is testing the case where the file is modified on the server
during the upload. So this test the precondition failed error.
The FakeGetReply logic was modified because resizing a 150MB big QByteArray
by increment of 16k just did not scale when downloading a big file.
2017-01-20 14:48:53 +01:00
Markus Goetz
9d7425b201 DB: Also migrate -shm and -wal to new name #3764 #5045 (#5459) 2017-01-16 15:42:11 +01:00
Christian Kamm
49f8143f00 Bugfixes for sync journal name generation and usage
* Use 'user' value for journal name generation
* Save journal name in settings
* Make owncloudcmd choose the right db
2016-11-23 16:48:15 +01:00
Christian Kamm
61b4da944c Split folders configuration locations for backwards compatibility 2016-11-23 16:48:15 +01:00
Christian Kamm
6fe1868693 Always migrate .csync_journal to ._sync_xxx 2016-11-23 16:48:15 +01:00
Christian Kamm
9641c7a1e7 Rename sync journal to ._sync_xxx.db.
The added underscore means that older clients will also ignore the
file.
2016-11-23 16:48:15 +01:00
Christian Kamm
3e59a9b316 Merge branch 'master' into dbjournal_per_account 2016-11-23 16:47:56 +01:00
Christian Kamm
db24f60ae3 License: Adjust license of GPLv2 source files to GPLv2+
See #5180
2016-10-25 11:06:54 +02:00
Christian Kamm
e020a5327e Force a remote discovery when upgrading to this version #5242
A remote discovery will fix up any lingering problems caused by #5190.
2016-10-18 11:21:01 +02:00
Olivier Goffart
f75106fd8e SyncJournalDB: check the return value of the prepare statements in checkConnect
Relates to #5199
2016-10-11 11:35:26 +02:00
Klaas Freitag
e1a48e3c33 Move the journal file name generation to the syncjournaldb class.
As requested by Olivier.
2016-10-10 16:59:17 +02:00
Klaas Freitag
ed6a708460 SyncJournalDb: Migrate the old csync journal to the new name.
Migration strategy to the new filename.
2016-09-29 16:41:11 +02:00
Jocelyn Turcotte
e974771796 csync: Use an explicit instruction for should_update_metadata
The current way of tracking the need to update the metadata without
propagation using a separate flag makes it difficult to track
priorities between the local and remote tree. The logic is also
difficult to logically cover since the possibilities matrix isn't
100% covered, leaving the flag only used in a few situations
(mostly involving folders, but not only).

The reason we need to change this is to be able to track the sync
state of files for overlay icons. The instruction alone can't be
used since CSYNC_INSTRUCTION_SYNC is used for folders even though
they won't be propagated. Removing this logic is however not possible
without using something else than CSYNC_INSTRUCTION_NONE since too
many codepath interpret (rightfully) this as meaning "nothing to do".

This patch adds a new CSYNC_INSTRUCTION_UPDATE_METADATA instruction
to let the update and reconcile steps tell the SyncEngine to update
the metadata of a file without any propagation. Other flags are left
to be interpretted by the implementation as implicitly needing
metadata update or not, as this was already the case for most file
propagation jobs. For example, CSYNC_INSTRUCTION_NEW for directories
now also implicitly update the metadata.

Since it's not impossible for folders to emit CSYNC_INSTRUCTION_SYNC
or CSYNC_INSTRUCTION_CONFLICT, the corresponding code paths in the
sync engine have been removed.

Since the reconcile step can now know if the local tree needs metadata
update while the remote side might want propagation, the
localMetadataUpdate logic in SyncEngine::treewalkFile now simply use
a CSYNC_INSTRUCTION_UPDATE_METADATA for the local side, which is now
implemented as a different database query.
2016-08-17 15:39:31 +02:00
Olivier Goffart
88e5a9411a SyncEngine: Reads the data-fingerprint property (#5056)
When it changes, assume a backup was recovered, and keep conflict files.

Issues: #2325 and https://github.com/owncloud/enterprise/issues/966
2016-08-02 10:30:49 +02:00
Klaas Freitag
37fc4e4332 SyncJournalDb: Cleanup of Constructor interface.
The parameter path is not longer needed.
2016-07-10 13:04:29 +02:00
Klaas Freitag
4ceee86c66 SyncEngine: Calculate a uniq journal name using the remote account.
That should allow to sync the same local folder to multiple remote destinations.

see #3764
2016-07-10 12:56:43 +02:00
Klaas Freitag
a1bc01d3b1 SyncJournalDb: Add a method to set the name of the db file. 2016-07-10 12:51:42 +02:00
Olivier Goffart
e960b265a8 Merge remote-tracking branch '2.1' into 2.2 2016-05-25 16:23:58 +02:00
Olivier Goffart
f6b35e5d58 SyncEngine: invalid the blacklist entry when the rename destination change
The problem in this case is if we rename the file "xxx" to "invalid\file".
The rename will fail because the new filename constains a slash, and it
will be blacklisted.
But then if the user re-rename the file to "valid_name", then we should
invalidate the blacklist entry and retry to upload. But we did not do
that because renaming don't change the mtime and we did not store the
rename target in the database

IL issue 558
2016-05-25 15:32:45 +02:00
Klaas Freitag
bf99306a53 Merge branch 'do_not_close_db_on_done' 2016-04-12 14:36:46 +02:00
Christian Kamm
41b950b7e6 Don't call reset() on a query after close() #4662 2016-04-12 10:30:46 +02:00
Klaas Freitag
868edb1f0d SyncJournalDb: Only close the db if the return code is not SQLITE_DONE
SQLITE_DONE is the indicator for not more query results, which is a legal
thing and not an error condition.

Also, check _getFileRecordQuery for null pointer, as close() wipes it.
2016-04-11 17:38:16 +02:00
Klaas Freitag
aaf43bd0d3 SyncJournalDb: Handle empty filename and avoid a bogus db query. 2016-04-11 17:36:15 +02:00
Klaas Freitag
648328fbe2 SyncJournalDb: Close the db on error in getFileRecord()
The idea is that the next call to any database operation will try to
reopen the database through the checkConnect() method. So even if there
was a disconnect trom the db file, this will reestablish the connection.
2016-04-11 12:40:19 +02:00
Klaas Freitag
7b1f02fcda SelectiveSync: Verify if the list could be read from journal.
If there is a read error from the database while trying to get
the list from database, make sure to not behave badly because
the list is empty.
2016-04-06 15:01:28 +02:00
Christian Kamm
4900703970 SqlQuery: Write NULL when intended #4548
In SQLite bindings are not cleared by sqlite3_reset() calls, so
skipping a sqlite3_bind call to create a NULL value doesn't work,
instead the previous value will be written.

To fix this, I clear all bindings in SqlQuery::reset and make sure
to explicitly bind NULL when desired in SqlQuery::bind.

To make sure there's no confusion about SqlQuery::reset and
sqlite3_reset, I rename our method to reset_and_clear_bindings().

(cherry picked from commit 7bd4f95b8c)
2016-03-15 15:17:35 +01:00
Christian Kamm
7bd4f95b8c SqlQuery: Write NULL when intended #4548
In SQLite bindings are not cleared by sqlite3_reset() calls, so
skipping a sqlite3_bind call to create a NULL value doesn't work,
instead the previous value will be written.

To fix this, I clear all bindings in SqlQuery::reset and make sure
to explicitly bind NULL when desired in SqlQuery::bind.

To make sure there's no confusion about SqlQuery::reset and
sqlite3_reset, I rename our method to reset_and_clear_bindings().
2016-03-15 14:38:31 +01:00
Olivier Goffart
cd83772112 SyncJounral: add an index on the error blacklist table
https://github.com/owncloud/enterprise/issues/1035
2016-02-11 14:38:38 +01:00
Christian Kamm
17895f3a30 Win: Fix filesystem detection for exFAT workaround #2701 2016-01-19 10:00:12 +01:00
Christian Kamm
a25f094c4c Checksums: Don't reupload if size and checksum are unchanged #3235
* Compute the content checksum (in addition to the optional
  transmission checksum) during upload (.eml files only)

* Add hook to compute and compare the checksum in csync_update

* Add content checksum to database, remove transmission checksum
2015-11-23 11:59:56 +01:00
Christian Kamm
25c177ca3b SyncJournalFileRecord: Remove unused mode field 2015-11-11 09:42:31 +01:00
Christian Kamm
17dd199cba Checksums: Treat more carefully in db #4034
In particular, preserve them on local rename or remote move.
2015-11-10 15:05:00 +01:00
Olivier Goffart
aaf16ff0e8 SyncJournalDB, clear all the queries before closing the database 2015-10-30 14:03:08 +01:00
Christian Kamm
496b1e907d Checksum: Don't lose it on metadata update #3735
Also improve tests.
2015-10-28 14:49:55 +01:00
Christian Kamm
b7823dc648 Checksum: Put checksum type into separate column #3735 2015-10-28 14:46:20 +01:00
Christian Kamm
dff37e11eb Propagate: Store computed checksums in db during upload #3735 2015-10-28 09:56:39 +01:00
Christian Kamm
60b2312ab6 SyncJournal: Add checksumHeader column to metadata table #3735 2015-10-28 09:56:39 +01:00
Olivier Goffart
597d36dcf2 csync_statedb: Use the index in csync_statedb_get_below_path
Make an index from the path, and make a query that uses the index
2015-10-21 16:38:26 +02:00
Olivier Goffart
df135a0bb2 Merge branch '2.0'
Conflicts:
	src/gui/folder.cpp
2015-10-19 10:57:37 +02:00
Markus Goetz
0d5f2b5d31 Propagator: Reset upload blacklist if a chunk suceeds
For #3934
2015-10-16 10:06:45 +02:00
Phil Davis
f0e17fd9c0 libtypos comment and message typos for master 2015-10-05 09:05:09 +05:45