The word ChatGPT has captured the imagination and the internet, but does ChatGPT truly know everything, is it truly AGI? To answer some of these questions and to formulate what testing a conversational agent trained on a large language model would look like, we had chatGPT take the wit test where we asked it riddles. Find out what it said. Quick introduction to Large Language Models, how they are trained and what can be improved. Slides and examples of the test associated with ChatGPT. Discussion of why ChatGPT fails with understanding contexts, doesn't do well at verbal math, doesn't know what a venn diagram is and has never heard an egg crack and what it means for the next generation of a conversational AI model. We will go deeper into how it perceives metaphorical language and convoluted relationships, we will also explore how the performance compares to that in the audience to have a good sense of what AGI means.

Description: Quick introduction to Large Language Models, how they are trained and what can be improved. Slides and examples of the test associated with ChatGPT. Discussion of why ChatGPT fails with understanding contexts, doesn't do well at verbal math, doesn't know what a venn diagram is and has never heard an egg crack and what it means for the next generation of a conversational AI model.

Aroma Rodrigues is a master's student at UMass Amherst. As a techno-activist she has been a part of many projects that promote diversity and inclusion. She believes that Automation is the path to Inclusion. In 2016, a teammate of her "Shoes for the Visually Impaired" project presented it at the FOSSASIA. She reads, writes and enjoys walking to explore places. She presently works in a financial services firm and believes that solving problems that she has would solve problems for a large chunk of the world. An ML enthusiast she has about 20+ Coursera Certifications with the respective project work to support her learning in that field. She presented a talk on “De-mystifying Terms and Conditions using NLP” at PyCon 2018 and a talk called “Propaganda Detection in Fake News using Natural Language Processing” at PyCon ZA 2019 in Johannesburg. She spoke on detecting gender roles based biases in school textbooks at PyOhio 2020.