ChatGPT vs. MagicAI: Which AI is the Best Teaching Assistant for Math/Science Educators?

I have been experimenting with ChatGPT this year to make lesson plans, generate assessments, and help brainstorm ideas in my role as a math/science teacher. Magic School AI is specifically geared towards teachers, so I thought it would be helpful to compare and contrast their functionality for someone in my role.

Set up and Premium vs Free Options

Both ChatGPT and Magic AI are easy to sign up for and have free and paid options. I signed up for both of them using my google account. Magic AI looks nicer and has built-in tools for specific teacher tasks. ChatGPT doesn’t have preset tools, but it can do all the same things right in the chat, and you can customize and adjust what it gives you until you have reached the daily limit, then it downgrades you to an earlier version. Magic AI, on the other hand, doesn’t let you customize things in the free version.

Making a Lesson Plan

Magic AI generates 5E lesson plans, which is not an acronym that I was familiar with, but after generation it seemed to be the same basic lesson plan style I was used to. The lesson it generated was fine, but I found it a bit frustrating that I wasn’t able to customize the lesson in the free version. I had to go to other tools to generate things like notes, worksheets, activities, etc that would pair with my lesson. ChatGPT can also generate a lesson plan. It didn’t follow the same acronym, but had the same general format of hook, instruction, practice, and then assessment. I liked that I could ask it to adjust what it generated, and also tell it to generate the notes, worksheet and activity page to go along with the lesson it planned. While I was doing this it did hit my daily limit, but that doesn’t stop it from working, you just end up with an older version that doesn’t give you as high of quality of materials. I think that the paid version of Magic AI can accomplish many of the same things, but ChatGPT is giving me all of that for free.

Making an Assessment

I have been using ChatGPT to generate assessments for the math classes I teach. Through experimentation I have learned that if you just tell it to generate a multiple choice test it can do that, but the questions will not include a lot of higher level thinking, and sometimes it will make mistakes when generating multiple choice distractors. For example, quadratic equations can have up to 2 solutions, and sometimes when generating answers it would just change the order of the solutions and think that these were different, but really a and c were the same answer. ChatGPT is more successful when you give it very specific prompts, for example instead of saying generate a test, you can improve the quality you get by telling it to generate a specific question type. For example, I would tell it to make a quadratic equation that is solved with the error of mixing up the positives and negatives in the factoring and have the students identify what error was made and then correct it. This made me a really good question, and once I have that prompt set up I can have ChatGPT generate several of these. My biggest issue with these questions is they don’t copy paste well, so I usually end up just screenshotting the questions instead of copying them into a word processor. This week I tried to see if MagicAI is better at this, and what I discovered was not at all. The basic version gave me fairly generic, simple questions, and because I was using the free version I was not able to do any tweaking to make it better. The program also seems to have some issues generating equations properly. I did like that magicAI allows me to directly upload a pdf of the curriculum I am working with.

Making Notes/Presentations

Magic AI has a feature that generates slides for teaching a concept. I decided to have it generate a 5 slide presentation for balancing chemical equations. ChatGPT doesn’t do this, but I tried to just directly copy and paste into google slides for comparison.

If you contrast these two, you will see that they both successfully input text onto the slide. Magic AI was a bit more seamless in that I didn’t have to copy and paste, but they both had some formatting issues. Magic AI didn’t maintain the subscripts in the chemical equations, and ChatGPT had an odd mix of equation and random symbols after the initial question. Both would require me to do quite a bit of editing for formatting and also to make them more visually appealing.

Generating Activities

Both Magic AI and ChatGPT can generate labs, and engaging activities, but I didn’t find these to be all that creative, and they were both pretty equivalent in this category. If you give them a generic prompt they will give you a lab that probably already exists in the teacher manual for your textbook. Because ChatGPT is more customizable I still prefer it, because I would rather be able to make my adjustments for free.

Student help/Tutoring

I expected this to be an easy win for Magic AI because it has this specific feature, but I actually found that ChatGPT was better at responding to my questions in a useful way. I tried to ask Magic AI questions about the quadratic formula in a way a student would phrase them, and I didn’t find it much more useful than looking at notes or a textbook. ChatGPT on the other hand took my initial prompt of “I know how to substitute into the quadratic formula, but I keep getting the wrong answer” and automatically generated a list of common mistakes students make and then offered to work through a question step by step.

In summary I think that ChatGPT is better than MagicAI at almost every task. I acknowledge that a paid version of MagicAI could be more useful, but I am cheap so I didn’t test it. I am not sure how it performs for humanities courses, and I did find the idea of student rooms interesting in that there was some tracking and control of what AI support students can use, but for math and science I didn’t really find it all that useful, and I will probably stick with ChatGPT.