The Department for Education have put out a call for evidence consultation on AI in education, with particular focus on ‘Generative artificial intelligence’, likely based on the popularity wave of ChatGPT. It can be found here – https://consult.education.gov.uk/digital-strategy/generative-artificial-intelligence-in-education/consultation/intro/ .

To help compose my own thoughts I’ve put together a SWOT analysis and a reflection based on my AI usage experience below.

Background:

There has been a significant push with regards AI in education and ChatGPT in particular. Some institutions have went with a push back, attempting to ban access on campus and there have been horror stories cropping up in the media of plagiarism and failed attempts to detect it due to human misconceptions of various systems. Other institutions have pushed forward, full steam ahead, looking ahead at how it can be embedded within curriculum and planning, but again empirically there seems to be human misconceptions permeating applications of ChatGPT and understanding the quality of what it can produce.

AI, and specifically prompt based generative models really jumped forward last summer. Open source DALL-E mini/Craiyon went viral and I got straight in on the fun; joyously prompting pictures of “Sauron on Saturday Kitchen”, “Hugh Fearnley-Whittingstall in a ball pit”, “Roy Cropper, Fast and the Furious” and “Vic Reeves and Bob Mortimer win the Crystal Maze” – showing them to my partner in hysterics, whilst she barely cracked a smile. Then when ChatGPT came along, I was asking it to craft arguments between Roy Keane and Micah Richards about which ‘Machiavellian Prince” Barry Chuckle is, where Roy Keane gets progressively angrier throughout. All being said though, there had to be more practical uses for this stuff than crafting flights of fancy around UK pop-culture references.

It became difficult to decipher what is genuine and what is ‘investor speak’ within the AI space. Is the “It’s coming soon” a genuine “coming soon”, or is it just venture capital and founder deflection from current inadequacies, or buzz building coupled with pivoted web3.0/NFT ‘influencers’ all aiming to drive value? I looked for academic journals, but the sheer quantity of the research in such a short time is over saturating – a lot which can currently seem quite ‘aimless’ and ‘unfocused’. Jack Clark of Anthropic, an AI safety and research company has a great newsletter called “Import AI” which is well worth a subscription/weekly read – https://importai.substack.com. The information and issues around synthetic data and the implications that go with that is very interesting.

Now, this is broader, but what if I want to think about education specifically? This UNESCO document with guidance for HE is fantastic at breaking everything down with AI – all the way to ethical considerations. The table on pg.9 and the figure 1 diagram are absolute must show to students at a number of levels, in my opinion – https://www.iesalc.unesco.org/wp-content/uploads/2023/04/ChatGPT-and-Artificial-Intelligence-in-higher-education-Quick-Start-guide_EN_FINAL.pdf 

I also stumbled on a great podcast I found talking to 3 authors of the book “AI in the Classroom: A complete AI classroom guide”. What I particularly loved about this was unlike a lot of the media presence, the ‘investor speak’ was very rolled back despite the overwhelming positivity of the three authors involved – https://www.coolcatteacher.com/ai-in-the-classroom-a-complete-ai-classroom-guide/ 

I don’t know if it was just the shock of hearing Dan Fitzpatrick’s North-East accent – and local affinity kicking in – but I feel the explanation of the current incarnation of AI ‘products’ essentially being at the Napster stage, when compared to the modern day streaming platforms, really resonated. It also mirrors a conversation I had with a colleague last week that it’s more likely we are at the Dogpile/AltaVista stage and the “Google” may not have reared its head yet, at all. 

I’ve re-linked to all of the things mentioned so far at the bottom of this post. I’ve also linked to ‘Personal Intelligence’ chat-bot HeyPi, which was recently launched by Inflection AI, which runs on its Inflection-1 language model. It is in early stages but this arguably provides the most scaffolded and ‘user friendly’ AI chat-bot experience from what I’ve used so far. However, it’s important to note that terms and conditions state 18+ for HeyPi currently, as opposed to the 13+ (with parental permission) for ChatGPT.

SWOT Pt.1:

Now onto what the post originally promised. I’ve focussed on Strengths and Weaknesses – as things stand in the current climate – based on my own personal experiences. They are summarised in the nice infographic, but more details are in the bullet points underneath.

STRENGTHS:

• Broader than generative models, but AI/machine learning already plays a big part in a number of assistive technologies eg. Screen readers, text-to-speech software. Beyond supporting SEND students in the class , more generally a number of online adaptive learning platforms exist, whilst a large portion of these use a programmed rules-based adaptive learning strategies, a few do have AI capabilities in some way.
• Great for quick text based reformatting and consolidating information, for example into a table for ease of consumption.
• Can offer direct student support – see page 9 of the UNESCO document for strategies.
• Can form terminology banks with translations for learners studying in a second language – through interrogation these can be made specific to your needs quite quickly.
• Generates multiple choice questions for a given topic area – but these can be a bit more difficult to interrogate to a suitable standard of curriculum challenge.
• Can be used to form exemplar essays/evidence of ‘student work’ for activities in class.
• Can be used to produce ‘logical’ (but fictional) large datasets for a contextual question.
• Can be told to answer as a certain person from history or literature using only their knowledge – may help support engagement in material for those more difficult to engage.

WEAKNESSES:

• Can produce errors or fake references (often related referred to as ‘hallucinations’).
• It lacks deep understanding of specific areas as well as Higher Order Thinking Skills. When trying to gain more depth through interrogation it instead just seems to shuffle things around.
• Accompanying that, it lacks the nuanced subject/content specific pedagogy that comes from knowledge of a discipline and reflection on teaching a topic over time.
• In my experience the multiple choice questions don’t interrogate to the standard suitable for specific curriculum needs and levels – not as well as terminology sheets do.
• When crafting exemplar essays/work you cannot get it to purposely include misconceptions you want students to identify in an example.
• Whilst it can produce datasets, it struggles to map to given circumstances as requested – for example it will not consistently produce a paired dataset for which evidence to reject null hypothesis in a repeated measures t-test would be significant to a 5% level, when given prompts in a variety of ways.
• For the last “Strength”, the chat bot loses track of timeline so will answer with information the person did not/could not know – also it would be an activity that would be ‘live’ in the classroom, preventing ability to audit for ‘hallucinations’.
• Even with elements of a specification loaded, it struggles to map to relevant curriculum knowledge. Submitting specification information as a prompt is impractical due to formatting and often too long for the chat to interpret (there are other methods of attempting to do this this which will be discussed in opportunities).

The 2nd part of this analysis which will focus on opportunities and weaknesses will be posted later. It will look more at the future, but also how humans interact with the current system. The second part will also have a final analysis and a personal conclusion.

Links and Further Reading:

OpenAI – ChatGPT – https://openai.com

HeyPI – https://heypi.com/talk

Craiyon – https://www.craiyon.com

Stable Diffusion – https://stablediffusionweb.com/#dem

UNESCO Quick Start guide to AI in Higher Education – https://www.iesalc.unesco.org/wp-content/uploads/2023/04/ChatGPT-and-Artificial-Intelligence-in-higher-education-Quick-Start-guide_EN_FINAL.pdf

30 minute podcast with the writers of “AI in the classroom…” –  https://www.coolcatteacher.com/ai-in-the-classroom-a-complete-ai-classroom-guide/

Import AI Newsletter –  https://jack-clark.net