Currently train.py is overloaded with many independent features.
Understanding the code and what will be the result of a training
call requires untangling the entire script. It's also an error
prone UX. This is a first step at separating independent parts
into their own scripts.
* Redo remote I/O changes once more; this time without messing with taskcluster
* Add bin changes
* Fix merge-induced issue?
* For the interleaved case with multiple collections, unpack audio on the fly
To reproduce the previous failure
rm data/smoke_test/ldc93s1.csv
rm data/smoke_test/ldc93s1.sdb
rm -rf /tmp/ldc93s1_cache_sdb_csv
rm -rf /tmp/ckpt_sdb_csv
rm -rf /tmp/train_sdb_csv
./bin/run-tc-ldc93s1_new_sdb_csv.sh 109 16000
python -u DeepSpeech.py --noshow_progressbar --noearly_stop --train_files ./data/smoke_test/ldc93s1.sdb,./data/smoke_test/ldc93s1.csv --train_batch_size 1 --feature_cache /tmp/ldc93s1_cache_sdb_csv --dev_files ./data/smoke_test/ldc93s1.sdb,./data/smoke_test/ldc93s1.csv --dev_batch_size 1 --test_files ./data/smoke_test/ldc93s1.sdb,./data/smoke_test/ldc93s1.csv --test_batch_size 1 --n_hidden 100 --epochs 109 --max_to_keep 1 --checkpoint_dir /tmp/ckpt_sdb_csv --learning_rate 0.001 --dropout_rate 0.05 --export_dir /tmp/train_sdb_csv --scorer_path data/smoke_test/pruned_lm.scorer --audio_sample_rate 16000
* Attempt to preserve length information with a wrapper around `map()`… this gets pretty python-y
* Call the right `__next__()`
* Properly implement the rest of the map wrappers here……
* Fix trailing whitespace situation and other linter complaints
* Remove data accidentally checked in
* Fix overlay augmentations
* Wavs must be open in rb mode if we're passing in an external file pointer -- this confused me
* Lint whitespace
* Revert "Fix trailing whitespace situation and other linter complaints"
This reverts commit c3c45397a2.
* Fix linter issue but without such an aggressive diff
* Move unpack_maybe into sample_collections
* Use unpack_maybe in place of duplicate lambda
* Fix confusing comment
* Add clarifying comment for on-the-fly unpacking
I usually do this as a conditional because tf takes a while to load and it's nice to skip it when you want to run a script that just preps data or something like that, but it doesn't seem like a big deal.
