[Mova](https://md.mova.elamur.ai/free/en/) · [this page in HTML](https://mova.elamur.ai/free/en/docs/papers/he-2021-speaker-turn-modeling-dialogue-act/)

[home](https://md.mova.elamur.ai/free/en/) / [docs](https://md.mova.elamur.ai/free/en/docs/) / [papers](https://md.mova.elamur.ai/free/en/docs/papers/) / [English](https://md.mova.elamur.ai/free/en/docs/papers/en/) / Speaker Turn Modeling for Dialogue Act Classification

# Speaker Turn Modeling for Dialogue Act Classification

**Authors:** Zihao He, Leili Tavabi, Kristina Lerman, Mohammad Soleymani · **Year:** 2021 · **Venue:** Findings of EMNLP 2021 **Link:** [aclanthology.org/2021.findings-emnlp.185](https://aclanthology.org/2021.findings-emnlp.185/) · **Licence:** [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)

[**34** citations](https://openalex.org/works?filter=cites:W3199809110) openalex · 2026-10-18[**12** citations in our library](https://md.mova.elamur.ai/free/en/docs/papers/cited-by/he-2021-speaker-turn-modeling-dialogue-act/) of the 140673 papers whose bibliographies we read[Findings of the Association for Computational Linguistics: EMNLP 2021](https://aclanthology.org/volumes/2021.findings-emnlp/)[licence](https://creativecommons.org/licenses/by/4.0/)

- [Publisher's page](https://aclanthology.org/2021.findings-emnlp.185/)
- [PDF](https://aclanthology.org/2021.findings-emnlp.185.pdf)
- [DOI 10.18653/v1/2021.findings-emnlp.185](https://doi.org/10.18653/v1/2021.findings-emnlp.185)
- [arXiv 2109.05056](https://arxiv.org/abs/2109.05056)
- [OpenAlex](https://openalex.org/W3199809110)

**Authors (4)**

1. [Zihao He](https://openalex.org/A5090904454) [ORCID](https://orcid.org/0000-0001-9052-6310) · h 7 · 276 citations · 49 works
2. [Leili Tavabi](https://openalex.org/A5021142881) [ORCID](https://orcid.org/0009-0003-5270-1984) · h 8 · 562 citations · 20 works
3. [Kristina Lerman](https://openalex.org/A5049634383) [ORCID](https://orcid.org/0000-0002-5071-0575) · h 59 · 18644 citations · 1837 works
4. [Mohammad Soleymani](https://openalex.org/A5024169758) [ORCID](https://orcid.org/0000-0002-5873-1434) · h 33 · 12652 citations · 134 works

Citations and author metrics from openalex (2026-10-18); authors with the same name are not always told apart.

## What it is about

The paper adds one bit of dialogue structure to a dialogue act tagger: whether the speaker changed. For multi-party talk the speakers are relabelled so that the label flips at each change of speaker (0,0,1,2,3,3,1 → 0,0,1,0,1,1,0), and two learned «speaker turn» vectors are added to RoBERTa utterance vectors before a bidirectional GRU over the conversation. Accuracy rises from 82.4 to 83.2 on Switchboard (43 acts) and from 90.7 to 91.4 on MRDA (5 classes); on DailyDialog, where speakers strictly alternate, the gain is small (86.9 → 87.5 with topic). Adding conversation topic helps DailyDialog but hurts Switchboard.

## Ideas

- I1. Add a binary «speaker changed since the previous quote» slot to `prag/src/feats.rs`, computed from the speakers that `prag/src/frames.rs` resolves for quotes (or from the narrator frame: *said the fox* vs *said the crow*). Today `quote=cont` marks continuation of one speech but not a change of speaker between two quotes. The paper's gain is from exactly this bit; measure act macro-F1, and agree/disagree and ask recall, in `prag eval`.
- I2. Combine it with `prev` as a pair feature (previous act × speaker changed) in the perceptron: «ask» followed by a change of speaker predicts an answer or refusal, without change it predicts more asking or explaining. Compare against the two features alone.

## For Mova

[38. Dialogue, discourse markers and reported speech](https://md.mova.elamur.ai/free/en/docs/languages/en/38-dialogue-and-discourse-markers/) §8 (dialogue in tales) and §11 (how Mova reads dialogue), `31-speech-acts.md` §9. Code: `prag/src/feats.rs`, `prag/src/frames.rs` (quote speakers). Measure: act macro-F1 and per-act recall in `prag eval`, cross-validated, with the shuffled-feature control.

Topics: [Speech act theory](https://md.mova.elamur.ai/free/en/docs/papers/topics/speech-act-theory/)

## Cite

The paper itself, not this card. BibTeX from the library's record of the work. [DOI 10.18653/v1/2021.findings-emnlp.185](https://doi.org/10.18653/v1/2021.findings-emnlp.185) · [arXiv 2109.05056](https://arxiv.org/abs/2109.05056)

```
@inproceedings{he2021speaker,
  title = {{Speaker Turn Modeling for Dialogue Act Classification}},
  author = {He, Zihao and Tavabi, Leili and Lerman, Kristina and Soleymani, Mohammad},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2021},
  year = {2021},
  doi = {10.18653/v1/2021.findings-emnlp.185},
  eprint = {2109.05056},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2021.findings-emnlp.185}
}
```

- [← previous Deep Semantic Role Labeling: What Works and What's Next](https://md.mova.elamur.ai/free/en/docs/papers/he-2017-deep-semantic-role-labeling-what/)
- [next → Gemination and degemination in English affixation](https://md.mova.elamur.ai/free/en/docs/papers/hedia-2019-gemination-degemination-english-affixation/)
- [↑ back to English](https://md.mova.elamur.ai/free/en/docs/papers/en/)

## References

29 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Chandrakant Bothe, Cornelius Weber, Sven Magg, and Stefan Wermter. 2018. A context-based approach for dialogue act recognition using simple recurrent neural networks. arXiv preprint arXiv:1805.06280. [↗](https://arxiv.org/abs/1805.06280)
2. Po-Chun Chen, Ta-Chung Chi, Shang-Yu Su, and Yun-Nung Chen. 2017. Dynamic time-aware attention to speaker roles and contexts for spoken language understanding. In 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 554–560. IEEE. [find](https://scholar.google.com/scholar?q=Po%2DChun+Chen%2C+Ta%2DChung+Chi%2C+Shang%2DYu+Su%2C+and+Yun%2DNung+Chen%2E+2017%2E+Dynamic+time%2Daware+attention+to+speaker+roles+and+contexts+for+spoken+language+understanding%2E+In+2017+IEEE+Automatic+Speech+Recognitio)
3. Zheqian Chen, Rongqin Yang, Zhou Zhao, Deng Cai, and Xiaofei He. 2018. Dialogue act recognition via crf-attentive structured network. In The 41st international acm sigir conference on research & development in information retrieval, pages 225–234. [find](https://scholar.google.com/scholar?q=Zheqian+Chen%2C+Rongqin+Yang%2C+Zhou+Zhao%2C+Deng+Cai%2C+and+Xiaofei+He%2E+2018%2E+Dialogue+act+recognition+via+crf%2Dattentive+structured+network%2E+In+The+41st+international+acm+sigir+conference+on+research+%26+devel)
4. Ta-Chung Chi, Po-Chun Chen, Shang-Yu Su, and Yun-Nung Chen. 2017. Speaker role contextual modeling for language understanding and dialogue policy learning. arXiv preprint arXiv:1710.00164. [↗](https://arxiv.org/abs/1710.00164)
5. In this paper, we propose a model for encoding speaker turn changes to tackle DA classification. Kyunghyun Cho, Bart Van Merriënboer, Dzmitry Bah-Specifically, we introduce conversation-invariant danau, and Yoshua Bengio. 2014. On the properties speaker turn embeddings and add them to utterance of neural machine translation: Encoder-decoder ap-embeddings produced by a pretrained language proaches. arXiv preprint arXiv:1409.1259. model. Such a simple yet scalable module can be easily added to other models to obtain signifi- Pierre Colombo, Emile Chapuis, Matteo Manica, Emmanuel Vignon, Giovanna Varni, and Chloe Clavel. cantly better results. Experiments on three datasets 2020. Guiding attention in sequence-to-sequence demonstrate the effectiveness of our method. For models for dialogue act prediction. In Proceedings future work, we will explore transformer encoders of the AAAI Conference on Artificial Intelligence, (Vaswani et al., 2017) instead of RNNs for encod-volume 34, pages 7594–7601. 2154 [↗](https://arxiv.org/abs/1409.1259)
6. Devamanyu Hazarika, Soujanya Poria, Amir Zadeh, Erik Cambria, Louis-Philippe Morency, and Roger Zimmermann. 2018. Conversational memory network for emotion recognition in dyadic dialogue videos. In Proceedings of the conference. Association for Computational Linguistics. North American Chapter. Meeting, volume 2018, page 2122. NIH Public Access. [find](https://scholar.google.com/scholar?q=Devamanyu+Hazarika%2C+Soujanya+Poria%2C+Amir+Zadeh%2C+Erik+Cambria%2C+Louis%2DPhilippe+Morency%2C+and+Roger+Zimmermann%2E+2018%2E+Conversational+memory+network+for+emotion+recognition+in+dyadic+dialogue+videos%2E+In+Pr)
7. Ryuichiro Higashinaka, Kenji Imamura, Toyomi Me-guro, Chiaki Miyazaki, Nozomi Kobayashi, Hiroaki Sugiyama, Toru Hirano, Toshiro Makino, and Yoshihiro Matsuo. 2014. Towards an open-domain conversational system fully based on natural language processing. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 928–939. [find](https://scholar.google.com/scholar?q=Ryuichiro+Higashinaka%2C+Kenji+Imamura%2C+Toyomi+Me%2Dguro%2C+Chiaki+Miyazaki%2C+Nozomi+Kobayashi%2C+Hiroaki+Sugiyama%2C+Toru+Hirano%2C+Toshiro+Makino%2C+and+Yoshihiro+Matsuo%2E+2014%2E+Towards+an+open%2Ddomain+conversationa)
8. Ryuichiro Higashinaka, Katsuhito Sudoh, and Mikio Nakano. 2006. Incorporating discourse features into confidence scoring of intention recognition results in spoken dialogue systems. Speech Communication, 48(3-4):417–436. [find](https://scholar.google.com/scholar?q=Ryuichiro+Higashinaka%2C+Katsuhito+Sudoh%2C+and+Mikio+Nakano%2E+2006%2E+Incorporating+discourse+features+into+confidence+scoring+of+intention+recognition+results+in+spoken+dialogue+systems%2E+Speech+Communicati)
9. Yangfeng Ji, Gholamreza Haffari, and Jacob Eisenstein. 2016. A latent variable recurrent neural network for discourse relation language models. arXiv preprint arXiv:1603.01913. [↗](https://arxiv.org/abs/1603.01913)
10. Dan Jurafsky. 1997. Switchboard swbd-damsl shallow-discourse-function annotation coders manual. Institute of Cognitive Science Technical Report. [find](https://scholar.google.com/scholar?q=Dan+Jurafsky%2E+1997%2E+Switchboard+swbd%2Ddamsl+shallow%2Ddiscourse%2Dfunction+annotation+coders+manual%2E+Institute+of+Cognitive+Science+Technical+Report%2E)
11. Harshit Kumar, Arvind Agarwal, Riddhiman Dasgupta, and Sachindra Joshi. 2018. Dialogue act sequence labeling using hierarchical encoder with crf. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32. [find](https://scholar.google.com/scholar?q=Harshit+Kumar%2C+Arvind+Agarwal%2C+Riddhiman+Dasgupta%2C+and+Sachindra+Joshi%2E+2018%2E+Dialogue+act+sequence+labeling+using+hierarchical+encoder+with+crf%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+In)
12. Ji Young Lee and Franck Dernoncourt. 2016. Sequential short-text classification with recurrent and convolutional neural networks. arXiv preprint arXiv:1603.03827. [↗](https://arxiv.org/abs/1603.03827)
13. Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky. 2017a. Adversarial learning for neural dialogue generation. arXiv preprint arXiv:1701.06547. [↗](https://arxiv.org/abs/1701.06547)
14. Ruizhe Li, Chenghua Lin, Matthew Collinson, Xiao Li, and Guanyi Chen. 2018. A dual-attention hierarchical recurrent neural network for dialogue act classification. arXiv preprint arXiv:1810.09154. [↗](https://arxiv.org/abs/1810.09154)
15. Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017b. Dailydialog: A manually labelled multi-turn dialogue dataset. arXiv preprint arXiv:1710.03957. [↗](https://arxiv.org/abs/1710.03957)
16. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [↗](https://arxiv.org/abs/1907.11692)
17. Libo Qin, Zhouyang Li, Wanxiang Che, Minheng Ni, and Ting Liu. 2021. Co-gat: A co-interactive graph attention network for joint dialog act recognition and sentiment classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13709–13717. [find](https://scholar.google.com/scholar?q=Libo+Qin%2C+Zhouyang+Li%2C+Wanxiang+Che%2C+Minheng+Ni%2C+and+Ting+Liu%2E+2021%2E+Co%2Dgat%3A+A+co%2Dinteractive+graph+attention+network+for+joint+dialog+act+recognition+and+sentiment+classification%2E+In+Proceedings+of+t)
18. Vipul Raheja and Joel Tetreault. 2019. Dialogue act classification with context-aware self-attention. arXiv preprint arXiv:1904.02594. [↗](https://arxiv.org/abs/1904.02594)
19. Sujith Ravi and Zornitsa Kozareva. 2018. Self-governing neural networks for on-device short text classification. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 887–893. [find](https://scholar.google.com/scholar?q=Sujith+Ravi+and+Zornitsa+Kozareva%2E+2018%2E+Self%2Dgoverning+neural+networks+for+on%2Ddevice+short+text+classification%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural+Language+Processi)
20. John R Searle, PG Searle, S Willis, John Rogers Searle, et al. 1969. Speech acts: An essay in the philosophy of language, volume 626. Cambridge university press. [find](https://scholar.google.com/scholar?q=John+R+Searle%2C+PG+Searle%2C+S+Willis%2C+John+Rogers+Searle%2C+et+al%2E+1969%2E+Speech+acts%3A+An+essay+in+the+philosophy+of+language%2C+volume+626%2E+Cambridge+university+press%2E)
21. Elizabeth Shriberg, Raj Dhillon, Sonali Bhagat, Jeremy Ang, and Hannah Carvey. 2004. The icsi meeting recorder dialog act (mrda) corpus. Technical report, INTERNATIONAL COMPUTER SCIENCE INST BERKELEY CA. [find](https://scholar.google.com/scholar?q=Elizabeth+Shriberg%2C+Raj+Dhillon%2C+Sonali+Bhagat%2C+Jeremy+Ang%2C+and+Hannah+Carvey%2E+2004%2E+The+icsi+meeting+recorder+dialog+act+%28mrda%29+corpus%2E+Technical+report%2C+INTERNATIONAL+COMPUTER+SCIENCE+INST+BERKELEY+)
22. Elizabeth Shriberg, Andreas Stolcke, Daniel Jurafsky, Noah Coccaro, Marie Meteer, Rebecca Bates, Paul Taylor, Klaus Ries, Rachel Martin, and Carol Van Ess-Dykema. 1998. Can prosody aid the automatic classification of dialog acts in conversational speech? Language and speech, 41(3-4):443–492. [find](https://scholar.google.com/scholar?q=Elizabeth+Shriberg%2C+Andreas+Stolcke%2C+Daniel+Jurafsky%2C+Noah+Coccaro%2C+Marie+Meteer%2C+Rebecca+Bates%2C+Paul+Taylor%2C+Klaus+Ries%2C+Rachel+Martin%2C+and+Carol+Van+Ess%2DDykema%2E+1998%2E+Can+prosody+aid+the+automatic+c)
23. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational linguistics, 26(3):339–373. [↗](https://doi.org/10.1162/089120100561737) [our card](https://md.mova.elamur.ai/free/en/docs/papers/prag-2000-stolcke-dialogue-act-modeling-automatic-tagging/)
24. Leili Tavabi, Kalin Stefanov, Larry Zhang, Brian Bor-sari, Joshua D Woolley, Stefan Scherer, and Mohammad Soleymani. 2020. Multimodal automatic coding of client behavior in motivational interviewing. In Proceedings of the 2020 International Conference on Multimodal Interaction, pages 406–413. [find](https://scholar.google.com/scholar?q=Leili+Tavabi%2C+Kalin+Stefanov%2C+Larry+Zhang%2C+Brian+Bor%2Dsari%2C+Joshua+D+Woolley%2C+Stefan+Scherer%2C+and+Mohammad+Soleymani%2E+2020%2E+Multimodal+automatic+coding+of+client+behavior+in+motivational+interviewing%2E+)
25. Leili Tavabi, Trang Tran, Kalin Stefanov, Brian Bor-sari, Joshua Woolley, Stefan Scherer, and Mohammad Soleymani. 2021. Analysis of behavior classification in motivational interviewing. In Proceedings of the Seventh Workshop on Computational Linguistics and Clinical Psychology: Improving Access, pages 110–115. [find](https://scholar.google.com/scholar?q=Leili+Tavabi%2C+Trang+Tran%2C+Kalin+Stefanov%2C+Brian+Bor%2Dsari%2C+Joshua+Woolley%2C+Stefan+Scherer%2C+and+Mohammad+Soleymani%2E+2021%2E+Analysis+of+behavior+classification+in+motivational+interviewing%2E+In+Proceedings)
26. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+neural+information+processi)
27. Yao Wan, Wenqiang Yan, Jianwei Gao, Zhou Zhao, Jian Wu, and S Yu Philip. 2018. Improved dynamic memory network for dialogue act classification with adversarial training. In 2018 IEEE International Conference on Big Data (Big Data), pages 841–850. IEEE. [find](https://scholar.google.com/scholar?q=Yao+Wan%2C+Wenqiang+Yan%2C+Jianwei+Gao%2C+Zhou+Zhao%2C+Jian+Wu%2C+and+S+Yu+Philip%2E+2018%2E+Improved+dynamic+memory+network+for+dialogue+act+classification+with+adversarial+training%2E+In+2018+IEEE+International+Con)
28. Bo Xiao, Dogan Can, James Gibson, Zac E Imel, David C Atkins, Panayiotis G Georgiou, and Shrikanth S Narayanan. 2016. Behavioral coding of therapist language in addiction counseling using recurrent neural networks. In Interspeech, pages 908– 912. [find](https://scholar.google.com/scholar?q=Bo+Xiao%2C+Dogan+Can%2C+James+Gibson%2C+Zac+E+Imel%2C+David+C+Atkins%2C+Panayiotis+G+Georgiou%2C+and+Shrikanth+S+Narayanan%2E+2016%2E+Behavioral+coding+of+therapist+language+in+addiction+counseling+using+recurrent+ne)
29. Lei Xie, Naicai Sun, and Bo Fan. 2014. A statistical parametric approach to video-realistic text-driven talking avatar. Multimedia tools and applications, 73(1):377–396. [find](https://scholar.google.com/scholar?q=Lei+Xie%2C+Naicai+Sun%2C+and+Bo+Fan%2E+2014%2E+A+statistical+parametric+approach+to+video%2Drealistic+text%2Ddriven+talking+avatar%2E+Multimedia+tools+and+applications%2C+73%281%29%3A377%E2%80%93396%2E)