[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/langstream-2024-turetzky-hebdb-weakly-supervised-dataset-hebrew/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [Modern Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/he/) / HebDB: a Weakly Supervised Dataset for Hebrew Speech Processing

# HebDB: a Weakly Supervised Dataset for Hebrew Speech Processing

**Authors:** [Arnon Turetzky](https://md.mova.elamur.ai/free/en/docs/papers/authors/arnon-turetzky/); [Or Tal](https://md.mova.elamur.ai/free/en/docs/papers/authors/or-tal/); Yael Segal-Feldman; Yehoshua Dissen; [Ella Zeldes](https://md.mova.elamur.ai/free/en/docs/papers/authors/ella-zeldes/); [Amit Roth](https://md.mova.elamur.ai/free/en/docs/papers/authors/amit-roth/); Eyal Cohen; Yosi Shrem; Bronya R. Chernyak; Olga Seleznova; Joseph Keshet; [Yossi Adi](https://md.mova.elamur.ai/free/en/docs/papers/authors/yossi-adi/) · **Year:** 2024 · **Venue:** arXiv:2407.07566 **Link:** [arxiv.org/abs/2407.07566](https://arxiv.org/abs/2407.07566) · library `langstream-2024-turetzky-hebdb-weakly-supervised-dataset-hebrew` · **Licence:** [arXiv — non-exclusive license to distribute](https://arxiv.org/licenses/nonexclusive-distrib/1.0/license.html) (scope: readonly) — summary only

[**4** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/langstream-2024-turetzky-hebdb-weakly-supervised-dataset-hebrew/) of the 140673 papers whose bibliographies we readarXiv:2407.07566[licence](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

- [Publisher's page](https://arxiv.org/abs/2407.07566)
- [PDF](https://arxiv.org/pdf/2407.07566)
- [arXiv 2407.07566](https://arxiv.org/abs/2407.07566)

**Authors (12)**

1. Arnon Turetzky
2. Or Tal
3. Yael Segal-Feldman
4. Yehoshua Dissen
5. Ella Zeldes
6. Amit Roth
7. Eyal Cohen
8. Yosi Shrem
9. Bronya R. Chernyak
10. Olga Seleznova
11. Joseph Keshet
12. Yossi Adi

Citations and author metrics from OpenAlex (); authors with the same name are not always told apart.

## What it is about

HebDB is a speech corpus of roughly 2,500 hours of spontaneous Hebrew: podcasts on many topics plus testimonies of World War II survivors, released both raw and pre-processed. The pipeline resamples to 16 kHz mono, cuts speech with a voice activity detector, and transcribes each segment with Whisper large-v2; text with non-Hebrew characters is dropped, numbers and dates are spelled out, and final letters are folded. A forced aligner (MMS, on romanised and diacritised text) gives each utterance a confidence score; a threshold of 0.3 keeps about 1,470 of 1,690 hours. Two baselines are trained on it, HuBERT (95M) and a Conformer (110M). On the FLEURS Hebrew test set, the filtered HuBERT with a language model reaches 33.6 WER, the Conformer 41.9, Whisper 1.55B without a language model 27.1, and MMS (1,107 languages) with one 40.0.

## Ideas

- I1. Keep a confidence score per item and train only on items above a threshold. Any weakly labelled Hebrew source that Mova might use (automatic transcripts, auto-parsed text) can be gated the same way, and the gate is tested on a gold subset. The gain here is small (HuBERT 33.9 to 33.6, Conformer 43.0 to 41.9 with LM), so the gate must be measured, not assumed.
- I2. Deterministic normalisation before the labels are used: spell out numbers and dates, drop items with off-script characters (about 1% of samples in this paper), and fold final letters to their regular forms. The paper found the final-letter folding helpful. Each rule should be a counted test: how many items it changes on a sample, and whether the change helps.
- I3. Report a yield table per source (hours kept against hours recorded). In this corpus one source fell from 492 h to 67.4 h after processing, which shows that source quality varies a lot. Mova should log the same yield for every new data source, so that a bad source is visible before it reaches training.

## For Mova

I1 is a rule for the weak-data side of [11. Modern Hebrew in Universal Dependencies](https://md.mova.elamur.ai/free/en/docs/languages/he/11-ud-analysis/) (how automatically produced trees are trusted). I2 goes into [2. Script and spelling](https://md.mova.elamur.ai/free/en/docs/languages/he/02-script-and-spelling/) (final-letter folding as a test) and into the script filter of `tools/langseg`. I3 is a reporting habit for every corpus we build. The paper's speech models are outside Mova's scope, which parses text only.

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [arXiv 2407.07566](https://arxiv.org/abs/2407.07566)

```
@misc{turetzky2024hebdb,
  title = {{HebDB: a Weakly Supervised Dataset for Hebrew Speech Processing}},
  author = {Turetzky, Arnon and Tal, Or and Segal-Feldman, Yael and Dissen, Yehoshua and Zeldes, Ella and Roth, Amit and Cohen, Eyal and Shrem, Yosi and Chernyak, Bronya R. and Seleznova, Olga and Keshet, Joseph and Adi, Yossi},
  year = {2024},
  eprint = {2407.07566},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2407.07566}
}
```

- [← previous Mevaker: Conclusion Extraction and Allocation Resources for the Hebrew Language](https://md.mova.elamur.ai/free/en/docs/papers/langstream-2024-shalumov-mevaker-conclusion-extraction-allocation-resources/)
- [next → Enhancing TTS Stability in Hebrew using Discrete Semantic Units](https://md.mova.elamur.ai/free/en/docs/papers/langstream-2024-zeldes-enhancing-tts-stability-hebrew-using/)
- [↑ back to Modern Hebrew](https://md.mova.elamur.ai/free/en/docs/papers/he/)

## References

36 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in neural information processing systems, vol. 33, pp. 12 449–12 460, 2020. [find](https://scholar.google.com/scholar?q=A%2E+Baevski%2C+Y%2E+Zhou%2C+A%2E+Mohamed%2C+and+M%2E+Auli%2C+%E2%80%9Cwav2vec+2%2E0%3A+A+framework+for+self%2Dsupervised+learning+of+speech+representations%2C%E2%80%9D+Advances+in+neural+information+processing+systems%2C+vol%2E+33%2C+pp%2E+12+449%E2%80%93)
2. P. K. Rubenstein, C. Asawaroengchai, D. D. Nguyen, A. Bapna, Z. Borsos, F. d. C. Quitry, P. Chen, D. E. Badawy, W. Han, E. Kharitonov et al., “Audiopalm: A large language model that can speak and listen,” arXiv preprint arXiv:2306.12925, 2023. [↗](https://arxiv.org/abs/2306.12925)
3. C. Wang, Y. Wu, Y. Qian, K. Kumatani, S. Liu, F. Wei, M. Zeng, and X. Huang, “Unispeech: Unified speech representation learning with labeled and unlabeled data,” in International Conference on Machine Learning. PMLR, 2021, pp. 10 937–10 947. [find](https://scholar.google.com/scholar?q=C%2E+Wang%2C+Y%2E+Wu%2C+Y%2E+Qian%2C+K%2E+Kumatani%2C+S%2E+Liu%2C+F%2E+Wei%2C+M%2E+Zeng%2C+and+X%2E+Huang%2C+%E2%80%9CUnispeech%3A+Unified+speech+representation+learning+with+labeled+and+unlabeled+data%2C%E2%80%9D+in+International+Conference+on+Machine)
4. V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2015, pp. 5206–5210. [find](https://scholar.google.com/scholar?q=V%2E+Panayotov%2C+G%2E+Chen%2C+D%2E+Povey%2C+and+S%2E+Khudanpur%2C+%E2%80%9CLibrispeech%3A+an+asr+corpus+based+on+public+domain+audio+books%2C%E2%80%9D+in+2015+IEEE+international+conference+on+acoustics%2C+speech+and+signal+processing+%28IC)
5. J. Kahn, M. Rivière, W. Zheng, E. Kharitonov, Q. Xu, P.-E. Mazaré, J. Karadayi, V. Liptchinsky, R. Collobert, C. Fuegen et al., “Libri-light: A benchmark for asr with limited or no supervision,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 7669–7673. [find](https://scholar.google.com/scholar?q=J%2E+Kahn%2C+M%2E+Rivie%CC%80re%2C+W%2E+Zheng%2C+E%2E+Kharitonov%2C+Q%2E+Xu%2C+P%2E%2DE%2E+Mazare%CC%81%2C+J%2E+Karadayi%2C+V%2E+Liptchinsky%2C+R%2E+Collobert%2C+C%2E+Fuegen+et+al%2E%2C+%E2%80%9CLibri%2Dlight%3A+A+benchmark+for+asr+with+limited+or+no+supervision%2C%E2%80%9D+in+)
6. C. Wang, M. Riviere, A. Lee, A. Wu, C. Talnikar, D. Haziza, M. Williamson, J. Pino, and E. Dupoux, “Voxpopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation,” arXiv preprint arXiv:2101.00390, 2021. [↗](https://arxiv.org/abs/2101.00390)
7. A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” in International Conference on Machine Learning. PMLR, 2023, pp. 28 492–28 518. [find](https://scholar.google.com/scholar?q=A%2E+Radford%2C+J%2E+W%2E+Kim%2C+T%2E+Xu%2C+G%2E+Brockman%2C+C%2E+McLeavey%2C+and+I%2E+Sutskever%2C+%E2%80%9CRobust+speech+recognition+via+large%2Dscale+weak+supervision%2C%E2%80%9D+in+International+Conference+on+Machine+Learning%2E+PMLR%2C+2023%2C+pp%2E)
8. V. Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandi et al., “Scaling speech technology to 1,000+ languages,” arXiv preprint arXiv:2305.13516, 2023. [↗](https://arxiv.org/abs/2305.13516)
9. Y. Zhang, W. Han, J. Qin, Y. Wang, A. Bapna, Z. Chen, N. Chen, B. Li, V. Axelrod, G. Wang et al., “Google usm: Scaling automatic speech recognition beyond 100 languages,” arXiv preprint arXiv:2303.01037, 2023. [↗](https://arxiv.org/abs/2303.01037)
10. C. Wang, S. Chen, Y. Wu, Z. Zhang, L. Zhou, S. Liu, Z. Chen, Y. Liu, H. Wang, J. Li et al., “Neural codec language models are zero-shot text to speech synthesizers,” arXiv preprint arXiv:2301.02111, 2023. [↗](https://arxiv.org/abs/2301.02111)
11. M. Le, A. Vyas, B. Shi, B. Karrer, L. Sari, R. Moritz, M. Williamson, V. Manohar, Y. Adi, J. Mahadeokar et al., “Voicebox: Text-guided multilingual universal speech generation at scale,” Advances in neural information processing systems, vol. 36, 2024. [find](https://scholar.google.com/scholar?q=M%2E+Le%2C+A%2E+Vyas%2C+B%2E+Shi%2C+B%2E+Karrer%2C+L%2E+Sari%2C+R%2E+Moritz%2C+M%2E+Williamson%2C+V%2E+Manohar%2C+Y%2E+Adi%2C+J%2E+Mahadeokar+et+al%2E%2C+%E2%80%9CVoicebox%3A+Text%2Dguided+multilingual+universal+speech+generation+at+scale%2C%E2%80%9D+Advances+in+n)
12. C. K. Reddy, H. Dubey, V. Gopal, R. Cutler, S. Braun, H. Gamper, R. Aichner, and S. Srinivasan, “Icassp 2021 deep noise suppression challenge,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 6623–6627. [find](https://scholar.google.com/scholar?q=C%2E+K%2E+Reddy%2C+H%2E+Dubey%2C+V%2E+Gopal%2C+R%2E+Cutler%2C+S%2E+Braun%2C+H%2E+Gamper%2C+R%2E+Aichner%2C+and+S%2E+Srinivasan%2C+%E2%80%9CIcassp+2021+deep+noise+suppression+challenge%2C%E2%80%9D+in+ICASSP+2021%2D2021+IEEE+International+Conference+on+Aco)
13. A. Defossez, G. Synnaeve, and Y. Adi, “Real time speech enhancement in the waveform domain,” arXiv preprint arXiv:2006.12847, 2020. [↗](https://arxiv.org/abs/2006.12847)
14. Y. Dissen, F. Kreuk, and J. Keshet, “Self-supervised Speaker Diarization,” in Proc. Interspeech 2022, 2022, pp. 4013–4017. [find](https://scholar.google.com/scholar?q=Y%2E+Dissen%2C+F%2E+Kreuk%2C+and+J%2E+Keshet%2C+%E2%80%9CSelf%2Dsupervised+Speaker+Diarization%2C%E2%80%9D+in+Proc%2E+Interspeech+2022%2C+2022%2C+pp%2E+4013%E2%80%934017%2E)
15. H. Yadav and S. Sitaram, “A survey of multilingual models for automatic speech recognition,” arXiv preprint arXiv:2202.12576, 2022. [↗](https://arxiv.org/abs/2202.12576)
16. T. Likhomanenko, Q. Xu, V. Pratap, P. Tomasello, J. Kahn, G. Avidov, R. Collobert, and G. Synnaeve, “Rethinking evaluation in asr: Are our models robust enough?” arXiv preprint arXiv:2010.11745, 2020. [↗](https://arxiv.org/abs/2010.11745)
17. L. Campbell, “Ethnologue: Languages of the world,” 2008. [find](https://scholar.google.com/scholar?q=L%2E+Campbell%2C+%E2%80%9CEthnologue%3A+Languages+of+the+world%2C%E2%80%9D+2008%2E)
18. A. Petrov, E. La Malfa, P. Torr, and A. Bibi, “Language model tokenizers introduce unfairness between languages,” Advances in Neural Information Processing Systems, vol. 36, 2024. [find](https://scholar.google.com/scholar?q=A%2E+Petrov%2C+E%2E+La+Malfa%2C+P%2E+Torr%2C+and+A%2E+Bibi%2C+%E2%80%9CLanguage+model+tokenizers+introduce+unfairness+between+languages%2C%E2%80%9D+Advances+in+Neural+Information+Processing+Systems%2C+vol%2E+36%2C+2024%2E)
19. A. W. Black, “Cmu wilderness multilingual speech dataset,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 5971– 5975. [find](https://scholar.google.com/scholar?q=A%2E+W%2E+Black%2C+%E2%80%9CCmu+wilderness+multilingual+speech+dataset%2C%E2%80%9D+in+ICASSP+2019%2D2019+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2E+IEEE%2C+2019%2C+pp%2E+5971%E2%80%93+5975%2E)
20. V. Pratap, Q. Xu, A. Sriram, G. Synnaeve, and R. Collobert, “Mls: A large-scale multilingual dataset for speech research,” arXiv preprint arXiv:2012.03411, 2020. [↗](https://arxiv.org/abs/2012.03411)
21. A. Conneau, M. Ma, S. Khanuja, Y. Zhang, V. Axelrod, S. Dalmia, J. Riesa, C. Rivera, and A. Bapna, “Fleurs: Few-shot learning evaluation of universal representations of speech,” in 2022 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2023, pp. 798–805. [find](https://scholar.google.com/scholar?q=A%2E+Conneau%2C+M%2E+Ma%2C+S%2E+Khanuja%2C+Y%2E+Zhang%2C+V%2E+Axelrod%2C+S%2E+Dalmia%2C+J%2E+Riesa%2C+C%2E+Rivera%2C+and+A%2E+Bapna%2C+%E2%80%9CFleurs%3A+Few%2Dshot+learning+evaluation+of+universal+representations+of+speech%2C%E2%80%9D+in+2022+IEEE+Spoken+La)
22. S. Izre’el, B. Hary, and G. Rahav, “Designing cosih: the corpus of spoken israeli hebrew,” International Journal of Corpus Linguistics, vol. 6, no. 2, pp. 171–197, 2001. [find](https://scholar.google.com/scholar?q=S%2E+Izre%E2%80%99el%2C+B%2E+Hary%2C+and+G%2E+Rahav%2C+%E2%80%9CDesigning+cosih%3A+the+corpus+of+spoken+israeli+hebrew%2C%E2%80%9D+International+Journal+of+Corpus+Linguistics%2C+vol%2E+6%2C+no%2E+2%2C+pp%2E+171%E2%80%93197%2C+2001%2E)
23. J. Azogui, A. Lerner, and V. Silber-Varod, “The open university of israel map task corpus (matacop),” 2016. [find](https://scholar.google.com/scholar?q=J%2E+Azogui%2C+A%2E+Lerner%2C+and+V%2E+Silber%2DVarod%2C+%E2%80%9CThe+open+university+of+israel+map+task+corpus+%28matacop%29%2C%E2%80%9D+2016%2E)
24. M. Marmorstein and N. Matalon, “The huji corpus of spoken hebrew: An interaction-oriented design of a corpus,” 2022. [find](https://scholar.google.com/scholar?q=M%2E+Marmorstein+and+N%2E+Matalon%2C+%E2%80%9CThe+huji+corpus+of+spoken+hebrew%3A+An+interaction%2Doriented+design+of+a+corpus%2C%E2%80%9D+2022%2E)
25. O. Sharoni, R. Shenberg, and E. Cooper, “Saspeech: A hebrew single speaker dataset for text to speech and voice conversion,” in Proc. Interspeech, 2023. [find](https://scholar.google.com/scholar?q=O%2E+Sharoni%2C+R%2E+Shenberg%2C+and+E%2E+Cooper%2C+%E2%80%9CSaspeech%3A+A+hebrew+single+speaker+dataset+for+text+to+speech+and+voice+conversion%2C%E2%80%9D+in+Proc%2E+Interspeech%2C+2023%2E)
26. Y. Marmor, K. Misgav, and Y. Lifshitz, “ivrit. ai: A comprehensive dataset of hebrew speech for ai research and development,” arXiv preprint arXiv:2307.08720, 2023. [↗](https://arxiv.org/abs/2307.08720) [our card](https://md.mova.elamur.ai/free/en/docs/papers/langstream-2023-marmor-ivrit-ai-comprehensive-dataset-hebrew/)
27. S. Team, “Silero vad: pre-trained enterprise-grade voice activity detector (vad), number detector and language classifier,” 2021. [find](https://scholar.google.com/scholar?q=S%2E+Team%2C+%E2%80%9CSilero+vad%3A+pre%2Dtrained+enterprise%2Dgrade+voice+activity+detector+%28vad%29%2C+number+detector+and+language+classifier%2C%E2%80%9D+2021%2E)
28. J. Sohn, N. S. Kim, and W. Sung, “A statistical model-based voice activity detection,” IEEE signal processing letters, vol. 6, no. 1, pp. 1–3, 1999. [find](https://scholar.google.com/scholar?q=J%2E+Sohn%2C+N%2E+S%2E+Kim%2C+and+W%2E+Sung%2C+%E2%80%9CA+statistical+model%2Dbased+voice+activity+detection%2C%E2%80%9D+IEEE+signal+processing+letters%2C+vol%2E+6%2C+no%2E+1%2C+pp%2E+1%E2%80%933%2C+1999%2E)
29. V. Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandi et al., “Scaling speech technology to 1,000+ languages,” arXiv preprint arXiv:2305.13516, 2023. [↗](https://arxiv.org/abs/2305.13516)
30. W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3451–3460, 2021. [find](https://scholar.google.com/scholar?q=W%2E%2DN%2E+Hsu%2C+B%2E+Bolte%2C+Y%2E%2DH%2E+H%2E+Tsai%2C+K%2E+Lakhotia%2C+R%2E+Salakhutdinov%2C+and+A%2E+Mohamed%2C+%E2%80%9CHubert%3A+Self%2Dsupervised+speech+representation+learning+by+masked+prediction+of+hidden+units%2C%E2%80%9D+IEEE%2FACM+Transactions+)
31. A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu et al., “Conformer: Convolution-augmented transformer for speech recognition,” arXiv preprint arXiv:2005.08100, 2020. [↗](https://arxiv.org/abs/2005.08100)
32. M. Ott, S. Edunov, A. Baevski, A. Fan, S. Gross, N. Ng, D. Grangier, and M. Auli, “fairseq: A fast, extensible toolkit for sequence modeling,” arXiv preprint arXiv:1904.01038, 2019. [↗](https://arxiv.org/abs/1904.01038)
33. A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber, “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in Proceedings of the 23rd international conference on Machine learning, 2006, pp. 369–376. [find](https://scholar.google.com/scholar?q=A%2E+Graves%2C+S%2E+Ferna%CC%81ndez%2C+F%2E+Gomez%2C+and+J%2E+Schmidhuber%2C+%E2%80%9CConnectionist+temporal+classification%3A+labelling+unsegmented+sequence+data+with+recurrent+neural+networks%2C%E2%80%9D+in+Proceedings+of+the+23rd+internat)
34. A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu, and R. Pang, “Conformer: Convolution-augmented Transformer for Speech Recognition,” in Proc. Interspeech 2020, 2020, pp. 5036–5040. [find](https://scholar.google.com/scholar?q=A%2E+Gulati%2C+J%2E+Qin%2C+C%2E%2DC%2E+Chiu%2C+N%2E+Parmar%2C+Y%2E+Zhang%2C+J%2E+Yu%2C+W%2E+Han%2C+S%2E+Wang%2C+Z%2E+Zhang%2C+Y%2E+Wu%2C+and+R%2E+Pang%2C+%E2%80%9CConformer%3A+Convolution%2Daugmented+Transformer+for+Speech+Recognition%2C%E2%80%9D+in+Proc%2E+Interspeech+20)
35. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017. [find](https://scholar.google.com/scholar?q=A%2E+Vaswani%2C+N%2E+Shazeer%2C+N%2E+Parmar%2C+J%2E+Uszkoreit%2C+L%2E+Jones%2C+A%2E+N%2E+Gomez%2C+%C5%81%2E+Kaiser%2C+and+I%2E+Polosukhin%2C+%E2%80%9CAttention+is+all+you+need%2C%E2%80%9D+Advances+in+neural+information+processing+systems%2C+vol%2E+30%2C+2017%2E)
36. C. Kim, D. Gowda, D. Lee, J. Kim, A. Kumar, S. Kim, A. Garg, and C. Han, “A review of on-device fully neural end-to-end automatic speech recognition algorithms,” in 2020 54th Asilomar Conference on Signals, Systems, and Computers. IEEE, 2020, pp. 277–283. [find](https://scholar.google.com/scholar?q=C%2E+Kim%2C+D%2E+Gowda%2C+D%2E+Lee%2C+J%2E+Kim%2C+A%2E+Kumar%2C+S%2E+Kim%2C+A%2E+Garg%2C+and+C%2E+Han%2C+%E2%80%9CA+review+of+on%2Ddevice+fully+neural+end%2Dto%2Dend+automatic+speech+recognition+algorithms%2C%E2%80%9D+in+2020+54th+Asilomar+Conference+on+Si)