The toolkit is an instant-messaging platform for conducting research on dialogue. The software consists of a chat server and chat clients, written in Java.
The chat-tool makes data-collection and preparation much easier and quicker. Configuration of the server and clients allows experimental designs that are much more standardized and reproducible. The toolkit can be used “off-the-shelf” to collect and prepare instant-messaging conversations. It is also a powerful, programmable toolkit for conducting experiments on live interaction.
All participants’ turns are routed via the server which can be scripted to detect, for example target words, phrases or syntactic constructions. This information can then be used, in real-time to trigger experimental interventions that are sensitive to the interactional context.
The toolkit provides an extensive API for scripting these manipulations, as well as a constantly expanding library of experimental setups that can be reconfigured.
The chattool automatically records all keypresses, words and turns, notifications (including typing notifications and read receipts), number of edits, typing speed, typing overlap.
Instead of costly and time-intensive transcription, all data is immediately available for analysis. The data is saved in a variety of formats - to help both qualitative and quantitative analyses (e.g. loading into SPSS, R, Excel, MATLAB).
The chat tool includes a set of dialogue tasks, including:
- The maze game
- The tangram task (which can be adapted to most other joint reference tasks)
- The confidence task
Other tasks can easily be programmed in java
Manipulating the interface
Existing software (e.g. whatsapp, viber, line, wechat) use subtly different interfaces that have undocumented, ad-hoc and constantly changing features. The effect of these interface features on interaction are currently poorly understood. The toolkit allows configuration of all aspects of the interface, including the screen dimensions, text (colour, font, positioning), typing synchrony and interleaving of turns, typing status, read receipts, as well as simulating network conditions, e.g. latency.
Manipulating the interaction
The real strength of this toolkit is in manipulating the interaction. Since all turns are relayed through the server, the server can identify and transform participants’ contributions. Possible kinds of experimental intervention include:
Conceptual & linguistic coordination
- Manipulation of lexical, syntactic and semantic constituents (e.g. by inserting “spoof” clarification requests into the interaction that target a specific element - see image below).
- Manipulation of the specificity of referring expressions.
- Manipulation of signals of positive feedback (e.g. inserting, blocking or transforming signals such as “ok”, “yeah”)
- Manipulation of signals of negative feedback (e.g. inserting, blocking or transforming clarification requests such as “what?” or “huh?” or “sorry why?”)
- Manipulation of discourse markers, e.g. “so?”
- Manipulation of priming & levels of alignment.
- Procedural coordination
- Manipulation of the timing and sequencing of turns.
- Manipulation of the timing and sequencing of turns.
Introducing artificial hesitations//disfluencies (e.g. “umm”, “uhh”)
- Group-membership and identity:
- Manipulation of participatory status (e.g overhearer vs. bystander).
- Manipulation of (apparent) identity of participant (e.g. gender, age, community membership)
- Assigning participants to different sub-groups, e.g. to allow group-specific conventions to emerge.
Download and run the toolkit
The latest version of the toolkit is available as a zip file
The source code of the project is hosted at https://github.com/dialoguetoolkit/chattool.
Mills, G. J. and Healey, P. G. T. (submitted) A dialogue experimentation toolkit.
Atkinson, M., Mills, G. J., & Smith, K. (2018). Social group effects on the emergence of communicative conventions and language complexity. Journal of Language Evolution.
Healey, P. G., Mills, G. J., Eshghi, A., & Howes, C. (2018). Running repairs: Coordinating meaning in dialogue. Topics in cognitive science, 10(2), 367-388.
Gao, G., Hwang S. Y., Jung M., & Fussell, S.R. (2018). Beyond Information Content: The Effects of Culture on Affective Grounding in Instant Messaging Conversations. CSCW 2018.
Yu, Y., Eshghi, A., Mills, G., & Lemon, O. J. (2017). The burchak corpus: a challenge data set for interactive learning of visually grounded word meanings. In Belz, Anya, Erkut Erdem, Katerina Pastra, and Krystian Mikolajczyk. "Proceedings of the Sixth Workshop on Vision and Language." In Proceedings of the Sixth Workshop on Vision and Language. 2017.
Liebman, N., & Gergle, D. (2016, February). It's (Not) Simply a Matter of Time: The Relationship Between CMC Cues and Interpersonal Affinity. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (pp. 570-581). ACM.
Concannon, S., Healey, P. G., & Purver, M. (2015). Shifting Opinions: An Experiment on Agreement and Disagreement in Dialogue SEMDIAL 2015 goDIAL, 15.
Nolle, Jonas, Kristian Tylen, and Gregory Mills. (2015) "Environmental affordances shape linguistic coordination in the maze game."
Jacobi, J., de Rechteren, A., Mills, G., & Redeker, G. (2015). Dutch-speaking children’s co-ordination skills in dialogue. Annual TABU Dag (Taalbulletin), Groningen
Engbrenghof, M., Mills, G., Redeker, G. (2015). Manipulating evidence of grounding in a collaborative communication game. Annual TABU Dag (Taalbulletin), Groningen
Mills, G. J. (2014) Dialogue in joint activity: complementarity, convergence, conventionalization New Ideas in Psychology.
Mills, G. J. (2012) Clarifying reference and plans in dialogue . Fifth International Conference of the German Cognitive Linguistics Association. University of Freiburg 10-12 October. Poster
Christine Howes. (2012) Coordination in dialogue: Using compound contributions to join a party. PhD thesis, Queen Mary University of London, 2012.
Mills, G.J. (2012) Making and breaking procedural conventions in dialogue . Poster presented at Annual meeting of the cognitive science society (CogSci), Kyoto
Davidenko, N. and Mills G. J. (2012) Describing faces: Conventionalizing ontologies through dialogic interaction . Poster presented at Annual meeting of the cognitive science society (CogSci), Kyoto
Christine Howes, Patrick G. T. Healey, Matthew Purver, and Arash Eshghi. (2012) Finishing each other's. . . responding to incomplete contributions in dialogue. In Architectures and Mechanisms for Language Processing, Riva del Garda, 2012.
Christine Howes, Patrick Healey and Matthew Purver (2012). Whose turn is it anyway? Same- and cross-person compound contributions in dialogue. Poster, in CogSci 2012, Sapporo, August 2012.
Christine Howes, Patrick G. T. Healey, Matthew Purver, and Arash Eshghi. (2012) Finishing each other's. . . responding to incomplete contributions in dialogue. In Proceedings of the 32nd Annual Conference of the Cognitive Science Society, Sapporo, 2012.
Mills, G. J. (2011) The emergence of procedural conventions in dialogue In Proceedings of the 33rd Annual Conference of the Cognitive Science Society. Boston. USA
Howes, C. Purver, M., Healey, P. G.T., Mills, G. (2011) On Incrementality in Dialogue: Evidence from Compound Contributions Dialogue and Discourse. Vol 2, No 1. Special Issue on Incremental Processing in Dialogue.
Gregoromichelaki, E., Kempson, R., Purver, M., Mills, G., Cann, R. (2011) Incrementality and intention-recognition in utterance processing Vol 2, No 1. Special Issue on Incremental Processing in Dialogue.
Patrick G. T. Healey, Arash Eshghi, Christine Howes, and Matthew Purver.(2011) Making a contribution: Processing clarification requests in dialogue. In Proceedings of the 21st Annual Meeting of the Society for Text and Discourse, Poitiers, July 2011.
Mills, G. J. and Gregoromichelaki, E. (2010) Establishing coherence in dialogue: sequentiality, intentions and negotiation. In Proceedings of SemDial (PozDial). Adam Mickiewicz University, Poznan, Poland 16-18th June.
Kempson, R., Gregoromichelaki E., Mills, G. Purver, M., Howes, C., Healey, P. (2010) On Dialogue Modelling, Language Processing Dynamics, and Linguistic Knowledge Linguistic Evidence 2010, Tübingen, Feb 2010.
Howes, C. , Healey, P. G. T., Mills, G. J. (2009) A: an experimental investigation into B:...split utterances In Proceedings of SIGDIAL 2009: the 10th Annual Meeting of the Special Interest Group in Discourse and Dialogue, Queen Mary University of London, Sept. 2009.
Kempson R, Gregoromichelaki E, Purver M, Mills G, Gargett A, Howes C (2009) How mechanistic can accounts of interaction be? In: Proceedings of Diaholmia, the 13th workshop on the semantics and pragmatics of dialogue
Mills, G. J. and Healey, P.G.T. (2008) Semantic negotiation in dialogue: mechanisms of alignment. in Proceedings of the 8th SIGdial workshop on Discourse and Dialogue, Columbus, OH, US; June 2008.
Mills, G. J. (2007) Semantic co-ordination in dialogue: the role of direct interaction. PhD Thesis.
Mills, G.J. and Healey, P. G. T. (2006) Clarifying Spatial Descriptions: Local and Global Effects on Semantic Co-ordination , In Proceedings of the 10th Workshop on the Semantics and Pragmatics of Dialogue. Potsdam. Germany.
Healey, P.G.T. and Mills, G.J. (2006) Participation, Precedence and Co-ordination in Dialogue. In Proceedings of the 28th Annual Conference of the Science Society. Vancouver. Canada.
Matthew Purver. The Theory and Use of Clarification Requests in Dialogue. PhD Thesis, University of London, 2004.
Healey, P.G.T., Purver, M., King, J., Ginzburg, J. and Mills, G. J. (2003) Experimenting with Clarification in Dialogue. in Alterman, R. and Kirsh, D. (eds) Proceedings of the 25th Annual Conference of the Cognitive Science Society. Mahwah, N. J.: LEA pp. 539-544.
Purver, M., Healey, P. G. T., King, J. Ginzburg, J. and Mills, G. (2003) Answering Clarification Questions. In Proceedings of the 4th SIGdial Workshop on Discourse and Dialogue, pp 23-33, Association for Computational Linguistics, Sapporo, Japan, July 2003.# Related toolkits https://arxiv.org/pdf/1812.03415.pdf https://link.springer.com/article/10.3758/s13428-017-0873-y https://dsg-bielefeld.github.io/slurk https://f.hypotheses.org/wp-content/blogs.dir/4280/files/2018/11/paper_14.pdf