mirrors/DeepSpeech
2
0
Fork 0
mirror of https://github.com/mozilla/DeepSpeech.git synced 2025-10-26 11:19:39 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,489 Commits 63 Branches 109 Tags 17 GiB
ftyers-patch-1
Commit Graph

70 Commits

Author SHA1 Message Date
Alexandre Lissy
0add08e30d Ensure python builds uses all ressources 2020-02-14 18:55:03 +01:00
Reuben Morais
8dedda7759 Address review comments 2020-02-11 19:44:36 +01:00
Reuben Morais
1d3b3a31a1 Address review comments and update docs 2020-02-11 19:44:36 +01:00
Reuben Morais
efbed73d5c Improve error handling around Scorer loading 2020-02-11 19:44:36 +01:00
Reuben Morais
1e2eb96248 Update all API consumers 2020-02-11 19:44:36 +01:00
Reuben Morais
ab08f5ee5a Change decoder API 2020-02-11 19:44:26 +01:00
Reuben Morais
16d5632d6f Write default values for alpha and beta into trie header 2020-02-11 19:44:26 +01:00
Reuben Morais
b33d90b7bd Load combined format from Scorer 2020-02-11 19:44:26 +01:00
Reuben Morais
214b50f490 Add generate_package tool to create combined scorer package 2020-02-11 19:44:26 +01:00
Reuben Morais
be2229ef29 Refactor Scorer so model/trie package can be created by an external tool 2020-02-11 19:44:26 +01:00
Reuben Morais
7c0354483e Stop including vocabulary data in LM.binary. 2020-02-11 19:44:19 +01:00
Alexandre Lissy
ff401732a3 Enforce CTC decoder version check 
Fix #2710
 2020-02-03 10:24:21 +01:00
Alexandre Lissy
f44d2ddeb9 Don't OOV_SCORE on empty prefix 
Fixes #2579
 2020-01-02 12:39:12 +01:00
Reuben Morais
d2eb305b73 Address review comment and add missing check for presence of scorer 2019-11-12 21:56:42 +01:00
Reuben Morais
0e6952c3a8 Avoid reconstructing strings twice on decode 2019-11-11 12:53:05 +01:00
Reuben Morais
c1b1a59423 Score prefix as soon as a grapheme is formed rather than 1 byte later 2019-11-11 12:52:48 +01:00
Reuben Morais
f4cdd988df UTF-8 target 2019-11-11 11:36:16 +01:00
Reuben Morais
ae705cf95f Remove first party code from common.a 2019-11-08 14:33:06 +01:00
Reuben Morais
8c82081779 Embed alphabet directly in model 2019-11-05 09:02:21 +01:00
lissyx
969146c908
Merge pull request #2462 from lissyx/py38 
Build Python 3.8 wheels
 2019-10-25 19:15:40 +02:00
Alexandre Lissy
f80dbcda75 Build Python 3.8 wheels 
Fixes #2461
 2019-10-25 12:28:27 +02:00
Reuben Morais
44a605c8b7
Merge pull request #2435 from mozilla/uplift-utf8-fixes 
Uplift general fixes from UTF-8 work
 2019-10-25 09:09:48 +00:00
Alexandre Lissy
ef3f8004ce Use std::shared_ptr instead of raw pointer for dictionary_ 
Fixes #2403
 2019-10-18 10:15:59 +02:00
Reuben Morais
31d81740ee Add debugging helpers to PathTrie 2019-10-15 12:49:38 +02:00
Reuben Morais
abc687b3b9 Specify BOS for final scoring at decode if applicable 2019-10-14 21:03:14 +02:00
Reuben Morais
c8802a38e7 Don't add special tokens to vocabulary 2019-10-14 21:03:14 +02:00
Reuben Morais
3015237e8d Replace incomplete sorts with partial sorts 2019-10-14 21:03:14 +02:00
Reuben Morais
aad1b2234b Debug build for ds_ctcdecoder package 2019-10-11 18:01:02 +02:00
Reuben Morais
67e3eefb95 Remove incorrect docs on Scorer 
X-DeepSpeech: NOBUILD
 2019-09-30 10:54:59 +02:00
Reuben Morais
ba56407376
Merge pull request #2383 from mozilla/scorer-cleanup 
Don't explicitly score the BOS token, and avoid copies when scoring sentences
 2019-09-27 11:06:24 +02:00
Reuben Morais
a323973521 Address review comments 2019-09-27 11:02:22 +02:00
Alexandre Lissy
86b44a7cb7 Load KenLM with LAZY 
Fixes #2384
 2019-09-26 21:08:27 +02:00
Reuben Morais
6dba6d4a95 Don't explicitly score the BOS token, and avoid copies when scoring sentences 2019-09-26 14:08:32 +02:00
Reuben Morais
0ac498bc50 Rename metadata probability field to confidence 2019-09-11 11:09:44 +02:00
Reuben Morais
89f63dcd69 Make Scorer init fallible and check it in callers 2019-08-26 13:59:15 +02:00
Reuben Morais
1b18494e63 Address review comments 2019-08-23 12:19:32 +02:00
Reuben Morais
b2ef9cca83 Make Alphabet init fallible and check it in model creation 2019-08-23 12:19:32 +02:00
Reuben Morais
4dabd248bc Make Alphabet copyable and default-constructable and avoid pointers 2019-08-23 12:19:32 +02:00
Reuben Morais
4d882a8aec Simplify decoder impl by making it object oriented, avoid pointers where possible 2019-08-23 12:19:32 +02:00
Reuben Morais
e3bf5d3cc6 Only update time step of leaf prefixes 
The intention of this check is to improve the accuracy of the timings by recording the time step where the character saw its highest probability rather than the first time step where it was seen. The problem happens when updating the time step of a prefix that already has children. In that case, if any of the children have a time step that is earlier than `new_timestep`, it'll break the linearity of the timings. My fix is to simply check that the prefix we're updating is a leaf.

For example, say during decoding we have the following beams (format is `(char | time)`, tree node id below, nodes with same id are the same object):

```
1. (-1 | 0 ) -> ('s' | 10) -> ('h' | 13) -> ('e' | 14)
        A                B                  C                D

2. (-1 | 0 ) -> ('s' | 10) -> ('h' | 14)
        A                B                  E
```

And the prefix list is [B, C, D, E]. Currently, if we process character 'h' in time step 15 with a probability higher than both C and E, we update both nodes to have time step 15, which breaks linearity in beam 1. With my fix, we only update node E, which is a leaf. In my tests this does fix the problem, but since we don't have any known good quality data to verify against, it's hard to know if it has other side effects.
 2019-08-20 12:03:59 +02:00
Alexandre Lissy
c3a2b6ea48 Ensure proper removal of prefix 
Fixes #2230
 2019-07-10 13:27:18 +02:00
Francis Tyers
6e53fd8fe0
typo 
X-DeepSpeech: NOBUILD
 2019-06-22 22:18:25 +01:00
Reuben Morais
31afc6811f Switch to ConstFst from VectorFst and mmap trie file when reading 2019-06-21 22:21:19 -03:00
Reuben Morais
95a54bfdb8 Modify copies rather than live prefix scores in decoder_decode 2019-06-19 19:39:57 -03:00
Reuben Morais
d5d55958da Clean up formatting in ctc_beam_saerch_decoder.cpp 2019-06-19 19:39:57 -03:00
Reuben Morais
3ceb05e8b9 Keep absolute per-stream time step in DecoderState 2019-06-11 07:58:16 -03:00
Reuben Morais
494b573c80 Address review comments 2019-06-04 11:58:38 -03:00
Reuben Morais
1c87bf781a Avoid sorting prefix array twice 2019-06-01 15:06:56 -03:00
Reuben Morais
a46288e1c8 Only create Output structure for beams that are returned in the API 2019-06-01 15:02:41 -03:00
dabinat
d9a269412e CTC beam search streaming decoder (+6 squashed commits) 
Squashed commits:
[2941b47] Fixed nits
[700572e] Restored old CTC decoder API
[5aaf75d] Fixed nits
[969d71a] Added a destructor for DecoderState
[af0be6e] Removed accumulated_logits
[9dcb7b4] CTC beam search streaming decoder
 2019-05-21 20:54:19 -07:00