Reflections from Constructing Modern Knowledge 2024
This article, written by Sugar Labs executive director Devin Ulibarri’s, details his experience at Constructing Modern Knowledge (CMK) 2024, a Constructionism institute for teachers, which was held over the summer this year from July 9–12.
Day one: Experiments with AI
I participated in a workshop on the first day of the conference. I selected the AI workshop, since we were working on so many AI projects for Sugar Labs this summer.
We did three main things. First, Gary Stager, the conference organizer and leader for this activity, led us in a challenge to create a program to test for, and then to generate, “hundred-dollar words” using Lynx.
Lynx is a web-based version of the LOGO programming language, the very first programming language for kids. “Hundred-dollar words” are words whose sum totals one-hundred, assuming that “A” is one, “B” is two, “C” is three, etc. Words like “pumpkin” (16 + 21 + 13 + 16 + 11 + 9 + 14 = 100) and “excellent” (5 + 24 + 3 + 5 + 12 + 12 + 5 + 14 + 20 = 100) are hundred-dollar words.
In our respective groups, we solved as many parts of the problem as we could, but it is challenging to construct logic that can detect hundred-dollar words, let alone generate hundred-dollar words, especially ones that are real words in the English language (Lynx does not have a dictionary). Our group did not complete the task, but we did learn a lot.
Note: Later, I did create a hundred-dollar word test in Music Blocks. You can see how it works here: https://youtu.be/zcB2rJML38s
The second activity was to create new projects with experimental AI integration within Scratch. We were given two main features that we could integrate into Scratch code. One is a block where you set the “tone” for the AI, and the other block takes input for a prompt.
I created a few projects with these new blocks. At first, I thought it might be nice to create a program that would improvise music, so I told it to choose random pitches within a certain key. The output that this generated was used as the value for a variable that was used as an input argument for the pitch of the melody. Another prompt was used to generate a random value for rhythm, which was then passed as an argument for the beat length. This was not very effective for two reasons. One reason is, as I found out, generative AI is not very good at generating random numbers. Another reason is that it took quite a while to generate each value, which greatly compromised the rhythm. After this, I used the blocks to generate poems and dialogue, and I found this to be a much more effective use of the AI feature.
After this, we did a third activity using short snippets of Python to create a few LLM-enabled projects. One was a single-bot chatbot, one was two chatbots talking to one another, which we embellished by having it converted into voice audio. We also did image interpretation and recipe creation. This was my first time doing any of this, so I learned a lot about what the possibilities are.
Working on my group’s project: Lego notation for the blind
The next day, we started our projects. The way that CMK is structured, we spend about fifteen minutes collecting various ideas from the entire group and writing them down on large pieces of paper. After we had about thirty ideas collectively as a group, we were asked to write our names on the papers with ideas that we’d like to work on. And once we had chosen the project that we most wanted to work on, we wrote it on a paper and advocated for it, trying to recruit team members.
I contributed a few ideas, and, when we went out to find groups, I was delightfully surprised to see that someone had adopted one of my ideas, which was to create a “Lego musical notation for the blind.” Jamie Chelel from MIT’s K12 Maker Lab and I paired up together and, in the spirit of “learn by doing,” got to work immediately.
Creating the basic features of the notation system didn’t take very much time at all. It’s pitch over time, using a large Lego base as a grid framework upon which to place rectangular Lego blocks to indicate the height (in pitch) and length of the notes (over time). After agreeing upon the basics, Jamie and I created ways to expand the register without a person getting lost by adding a block to mark a particular pitch similar to how a clef works (e.g. G clef encircling the pitch of G4).
The next steps were more difficult. The idea of CMK is to have teachers work on problems and solve real-world problems, learning things by inventing solutions. Coming up with a system is important, but the real power of the system is in its implementation. From here, we did our best to take that system and put it into the computer, where a person could take their notated music and do other things with it, such as play it back, export it as MIDI, or import it into something that allows for combining music with computation such as Music Blocks or Strudel.
Get Sugar Labs’s stories in your inbox
Join Medium for free to get updates from this writer.
The next two steps as I saw them were to 1) transfer the image into the computer; and then 2) use the data imported into the computer to export into some other system, such as ABC notation, MIDI, or directly into Music Blocks code. Towards these goals, we then began to experiment with various solutions. I’ll admit that I haven’t used Scratch very much. (I’ve probably used it for a total of five hours or so before the conference.) But since I had time, and since one of the CMK instructors mentioned that Scratch has pixel color detection from images, I decided to try Scratch in order to transfer the image to the computer.
The first thing I learned was that Scratch’s pixel color detection functionality was a little bit more nuanced than we originally thought. We tried to use an image both as an avatar and as a background, but neither of those methods worked. The glyph (i.e. the cat) was not able to detect the pixels from either the background or from another glyph’s costume. Then we tried color detection from a live feed from the laptop’s web camera. This worked, so I then went on to script it so that the glyph would run across the screen detecting blocks, first vertically to find the pitches, then horizontally, to find the placement of the notes over time. I wanted to make the program as robust as possible so I scripted it so that you could have a pitch space with any arbitrary number of pitches (in your mode or scale). We were able to get a prototype working for this the first day, but it was difficult for me to assign meaningful pitches to each of the glyphs. I ultimately wasn’t able to solve that problem within Scratch, even by the end of CMK.
I then started working on the data interpretation and export, this time using Music Blocks. Although constructing this took me the rest of the conference, I did get a rather satisfying prototype in place that can take a virtual grid of LEGO notation, perform it, and export into Music Blocks code within the program itself. Music Blocks has a lot of features for expressing musical ideas, so it was easier to, for example, specify pitches only in a particular key. Music Blocks also has ways to name glyphs, which is helpful for generating and using an arbitrary number of glyphs in a meaningful way. Music Blocks also has ways to create new variables on the fly, which can be helpful in these sorts of instances as well.
Music Blocks does not currently have a way to detect pixels from an image, either from a live camera feed or from avatars (i.e. analogous to “costumes” in Scratch) or stamps (i.e. analogous to “stamps” in Scratch)
On Thursday, Senior Director for pK-12 at MIT Open Learning and Sugar Labs board member Claudia Urrea came and saw our project. She had the great idea to construct a chassis that would house the Legos in a way that the camera would always be pointing at it. Until this point, we were demonstrating how the Scratch part of this (the mockup of inputting it into the computer) worked by holding up the board to the camera. Claudia’s idea was a simple but powerful improvement to our overall design.
In terms of our project, that’s about as far as we were able to take it. We created the specifications for a Lego music notation, a way to detect Legos in Scratch (but not to robustly utilize the detected data), and a way to robustly interpret an abstracted Lego music notation in Music Blocks (but no way yet to import from a camera into Music Blocks). The next biggest step would be to implement a feature into Music Blocks that would detect pixels from an image. With this feature, you could perform the legos detected on the screen and save it into another format, such as Music Blocks code, ABC, or Lilypond.
Takeaways
As for the conference itself, I had two main takeaways from being immersed in Constructionism, together with other Constructionist teachers
First, I learned a little bit more about being an “inventor.” It’s no secret that I didn’t graduate from a school like MIT. I studied music in college. Although computers have been a part of my life since I was relatively young, and I did enjoy science and math growing up, I didn’t formally study computer science or engineering. That said, the idea to “invent to learn” makes perfect sense to me, and I did take the conference as an opportunity to experience the inventor mindset. Of course, having been a part of the Sugar Labs community for nearly a decade has helped me to learn how to better break down larger problems into smaller, more achievable tasks, as well as taking moments to reassess the problem and approach it from different angles.
Second, I very much felt that educators are thirsty for opportunities to be Constructionist themselves–to feel the joy and thrill of project-based learning for themselves. It was a joy to be able to spend time with so many other passionate educators, who remain playful and deeply caring in their respective crafts. I will say that it seems like a shame that this year will be the final year of CMK. I think something like this should be expanding instead of contracting, although I could see the challenges that one would face in implementing such as this.
Conclusion
CMK was a fun conference. I learned a lot from being immersed together with many other Constructionist educators.
As for Sugar Labs, I think one thing we can be incredibly proud of in terms of Constructionist education is that — and I can say this with much more confidence after having attended this conference — we are very, very committed to Constructionism on many levels. We give students at various levels of their journey the agency to participate in hard and fun projects. Evidence of this is the fact that Music Blocks, for example, has over two hundred contributors. This shows that we work very hard to help new contributors find points of collaboration, while mentoring them to do their best.
Sugar Labs is also very dedicated to keeping teachers involved in the development process, as evidenced over the summer by the work of Sugar Labs board member and Sugarizer maintainer Lionel Laské. For all of these reasons and more, I can say that we have a very potent alchemy for Constructionist learning that I’d expect more and more educators and learners would want to participate in.
