=============== Week 15: =================

Entry 5: -----------------------------------------------------------------

Date: April 25th
Start Time: 11:30am
Duration: 6 hours

Went back into lab after a quick nap. Rewrote my main integration code into a version with just USB UART, the servos and the keypad for testing, had errors with keypad, added timer, had errors with servo, added extra counter initialization lines and tweaked timer values, ran USB UART Keypad Servo, issues with garbage values, picked first and last drums, servos not swinging, fixed timer issue with servo 2, servo 1 still not working for some reason, moved to let Adhiksit test LCD keypad so that we could do full integration, went through and fixed some LCD keypad errors, fully integrated, keypad and beat editor worked perfectly after minor debugging, Alan found hardware issue with servo, tried MIDI again and it worked, upload finished early every once in a while and faulted other times, fixed by adding bigger delay before sending on python side so that buffer is flushed before send, got working after each power on reset, presented. After the long night last night, I took a quick nap and then came back into lab to finish off integration before our demo. My full integration code was structured such that when the LCD functions were filled in it would be testable and should work. Due to this, I wanted to test the LCD functions with the keypad first before bringing the code in, but something changed since we left the lab 4 hours ago and the LCD wouldn't power on. Juho and Alan took a look at it and rewired the setup to fix it. While they were doing that, I wanted to split my current full integration code into subsections for partial integration. We had already tested USB UART alone, Servo keypad and LCD alone on the PCB so the only important ones left were Servo Keypad USB UART and LCD Keypad. Adhiksit had the LCD functions so I worked on the Servo Keypad USB UART integration. I already had the code from full integration so I made a new project in CUBE MX without any of the LCD components, copied over my full integration code and updated it to remove all beat editor and LCD functionality. I still wanted the buttons to match what I use in the actual integration so I changed the state machine to just have IDLE, Upload, play and play again states, having the B button being pressed twice transition from IDLE to upload, the * button go from download to play after a successful upload, # to end playing early and go to the play again state and * to go back to play. Alan was testing his variable hit intensity while I was writing this up so once he was done I added that code as well. I also removed all the beat editor beat table conversions and play functionality as well as the incomplete tempo code to keep it simple. Once it was done, it looked like below, with the Upload_MIDI and Start_Beat_Timers having the same logic as before and TIM7 being the keypad interrupt timer.

I tested it on the PCB and ran into a few errors so me and Alan took a look at the code. First off, it seemed like the keypad inputs weren't registering so I added a breakpoint and saw that it indeed never reached any of the keypad interrupts. We went through the code and the timer intiialzation and saw that I forgot to actually start the keypad timer. After fixing that and running it again, it looked like the USB UART ran properly, but the servos weren't swinging. After a bit of debugging, we saw that the USB-UART was getting a few weird values. It got through my test checks so it was definetely reading whatever values were being transmitted, but for some reason the num_notes was getting garbage for a few of the drums. After a bit of testing and looping through all notes, I saw that the values for the 1st and last drums always seemed to be perfect so for now we use those. We ran it again and it still wouldn't swing. By adding a breakpoint right before starting the timer, we saw that the input variables were all perfect so this was most probably something with the timers. We went through the intiialzation of all the timers and matched it with the working keypad servo code and found a missing line in the timer intiialzations, __HAL_TIM_URS_ENABLE, so we added it. Running it again, servo 2 worked perfectly, but servo 1 still wouldn't swing. After we quadruple checked all the setup and the timers, we decided that it was most probably because of the servo itself and/or its cable so we replaced those and let Adhiksit test the LCD code in the meantime.

After testing and debugging the LCD keypad integration (it had a lot of the same errors as the USB UART Servo Keypad integration), we connected everything and started testing full integration. Alan had been looking at the servo and found that it may have been too close to the drum to mechanically be able to swing so after recalibration we tested full integration. There were some minor build errors and changes, but we all made sure to note down all changes we had to make when modularly integrating so I had a list of the exact changes I needed to make to the main file. After making sure to actually call the intiialzation functions in main and that I had the latest versions of all the code, I ran it and it worked well. There was a minor issue with the init state not going to the beat editor, but once I fixed that it worked. We tested the beat editor side of things and the whole flow was perfect. Cells could be assigned different intensity, pages wrap around correctly, going back to a page with information retains it and both servos actually play this time. It can also go back to the editor after playing and the reset works well. Since the beat editor side was perfect, we moved on the MIDI side. The first time we tried it it worked perfectly and played the song on both servos which was amazing, but when we power cycled and tried again the microcontroller seemed to exit the upload midi section before all the data was sent. I thought this could be because it retained information in the receive buffer from previous runs so I added a time.sleep on the python side so that it could be run first, clear the buffers and then the upload midi can be called to actually get locked on a receive. This seemed to fix it since the code we flashed onto the PCB was able to successfully upload and play the song after every power cycle so I'm still not entirely sure why it crumbled like that during the final presentation.

Entry 4: -----------------------------------------------------------------

Date: April 24th
Start Time: 5pm
Duration: 13 hours

This was a very long day. I started by looking through Adhiksit's current LCD code and helping him debug and list out all the functions he'd need for the current system design. He was running into some issues with text and after looking through the manual and from yesterday's work with the LCD, seemed to stem mainly from the fact that the pixel assigned as the starting point of a letter is the top left of the letter and letters have different pixel widths so getting the correct spacing between letters is more of an art than a science and needed a lot of trial and error. The main functions I identified that he would need to add were one to draw a grid with the initial labels, one to fill a specific cell number for changing intensity, one to fill the border of a cell a specific color to indicate whether it's selected or deselected and a welcome screen for now. The plan was that I'd work on full integration code while he implements these functions and any functions I think I'd need, I'd make calls to in my code and ask him to write a function up for it. I have worked with the LCD a decent amount this week so I have a pretty good idea of what it can and can't do with ease so I used that to split the burden between LCD_Driver functions and the main system functionality.

After this, Juho finished the flywiring to connect the extra UART7 RX pin to TX of the FT232RL device and the TX pin to the RX of the FT232RL. Originally we had planned to use the integrated USB-UART chip on the PCB, but after the errors from the past few days, we decided to cut our losses and just use the FT232RL device directly since the IC's datasheet wasn't great and getting it to work would involve complex flywiring that we still aren't sure would actually fix it. After this was complete I connected my laptop and loaded up my USB UART PCB code I had set up previously to test it. Since this was the same part I had just gotten working to prototype a few days back, the code worked perfectly and sent the data through USB from my script and received it through UART on the PCB. We called over a TA for checkoff and he asked for a demo on how the code could be used to swing the drum by using an LED. I intially thought he meant to remake the project with timers added and then write up some code to blink the LED at each point the timer is hit, but after asking him for clarification and discussing with Juho, I saw that he just meant to use HAL_Delay to stall between times and blink the LED accordingly. Since he had to go soon, I wrote up some quick code to blink the LED at the appropriate rate by just calculating the duration to the next hit, blinking the LED and then HAL_Delaying for the time leftover, while skipping any extra 0 entries. This worked and flashed the LED at the beat of the track so we got the PSDR checkoff.



With the checkoff out of the way and a general plan for the system in mind, I started working on full integration of all components. I started by making a new project in CUBE MX. I had Alan's Servo Keypad PCB integration as well as my USB UART PCB integration and the initial working LCD PCB code so I opened all of those CUBE MXs and made sure to add every component from all the projects as well as all their timers and interrupts. After checking it with the datasheet, the three other configurations and the rest of the team, I generated the starter code and began working on the system code. I based the code off of the servo keypad integration which essentially just has interrupts and hits a specific if/else if statement for any button pressed so there's a block for everything to do when 0 is pressed, another for 1, etc. To start off with, the current code only had 0 for edit/play and 1 for tempo change mode so I started by adding in an initial IDLE state where it can then go into either editor or upload. I spent some time going over my idea for the entire system and what I wanted each button to do. My main vision for the system was a grid in editor mode where each cell has a cell number and we have a page number telling us what page we're on. Since * was being used for play and # for tempo change, I wanted 7, 8, 9 and 0 to act as arrow keys around those to move around in the editor. With that in mind, I wanted C and D to control intensity up and down, foregoing the selection of a cell before updating it and instead keeping it quick and fluid. This left a good number of keys free so I assigned A as a hard reset, B as the upload, 4 as page left and 6 as page right. I started writing up the code with this vision in mind, key by key. The arrow keys of 7, 8, 9 and 0 weren't too bad, with the main challenge being wrapping to new pages and updating the LCD. My idea was that when moving to a new cell, we first use the LCD_border command to fill the current cell the background color since we're not selecting it anymore and the new cell needs to be bordered the selection color. For wrapping pages, my solution was essentially that I will have a hit_intensity table for every instrument which is an array of all the hit intensities from the start. Whenever a cell is changed in intensity, I would have to change the hit_intensity table at the index based on the current page and column so if it was on page 0 and column 15, that would correspond to index 15 of the table, while page 3 column 0 would correspond to index 48. If the table is correct, then whenever I go to a new page, I can just loop through the necessary 16 indices of the hit_intensity tables and fill the cells with the appropriate colors, while also clearing the other cells, deselecting the current cell and selecting the new one. After adding in the code for the arrow keys (7, 8, 9 and 0), the intensity changers (C and D) and the page changers (4 and 6), Adhiksit made a big discovery on the LCD side. He found that the LCD actually doesn't retain what it is currently flashing on the screen when you make a new display list. This essentially means that the current way I was using the LCD functions by building the grid once and then writing over whatever needs to change wouldn't work. After discussing and thinking it over, we found that this would actually make the usage a lot easier. On Adhiksit's end, most functions would just add to the display list and wouldn't actually complete it. I came up with a function called LCD_Current_Grid() which essentially just updates the entire screen based on the current values. It looks through the hit_intensity_table and fills all cells as needed, looks at the current cell and selects it and looks at the current bpm and page number and displays them so it could essentially just be called after any of these variables are changed.

After changing my code to match this new format, I started adding in the MIDI upload functionality to the design. I changed the state machine to 6 states by seperating the play and edit states and adding upload and play again states. I added in my code from my USB UART PCB project and thought of a few more LCD functions I wanted Adhiksit to make. The first was a success screen after a successful input, the next was an error screen after anything went wrong, after that was an upload midi file screen and the last two were a currently playing sreen and a play again screen. With these my state machine should be in a good place. I also started adding to the function which actually plays the servos. Currently it just initializes and sets its values, uses its time arrays to create gap arrays and then starts the timers. I need the actual array to be either the time array extracted from the hit_intensity table or some instruments extracted from the MIDI files. For the MIDI upload, they're in the same format so the work is minimal, but for the beat editor, we need to convert it into an array of times to strike the drum. Since each time step in the editor is 0.25 seconds at 60bpm, I can just accumulate this value for all 0's in the hit intensity_tabke and then the stored up time adds up with the previous beat to become the next time stamp to swing.

I discussed my current code with Alan and we went over some functionality that is still missing from the program. THe first was to be able to stop while playing. This was pretty easy since stopping the beat just meant disabling the timers and going to the play again state so I assigned the functionality to #. The next was to correctly calculate the lengths of the actual beat arrays being used so that the timers know when to stop for both MIDI uploads and beat editor which wasn't too bad. In the end, the main state machine and keypad section looked like below.

With this in good shape, I can come in tomorrow to hopefully get full integration tested before the final review.

Entry 3: -----------------------------------------------------------------

Date: April 23rd
Start Time: 8:00pm
Duration: 7 hours

I planned to come in at night and work for a few hours on integration and showing Adhiksit how the LCD stuff works so that he could finish the GUI since I thought we were in a pretty good place after the LCD worked yesterday, but I came in a bit early after seeing the team message about how bad things were going. The team's morale was really low and after discussing it with everyone, it looks like the PCB fried and the pads on the microcontroller melted between when I got the LCD working yesterday and now. To add insult to injury, the pictures I sent of the working LCD were apparently not enough to get checkoff for the SPI PSDR since the TAs wanted to see it cycle between multiple screens. Juho was resoldering the PCB to get it working again and Alan was working on getting tempo control working and cleaning up the keypad code so me and Ahiksit wanted to get my LCD code working on the nucleo board so that we could have somewhere to test the GUI code. I tested my code yesterday directly on the PCB since I had set it up there so I started by double and triple checking all the wiring on the nucleo board with Adhiksit. I checked this by comparing the physical wires to the manual's nucleo pins to GPIO pins mapping to get the exact current pins being used for all the SPI signals, reset and chip select. Everything looked fine so I made another new project for the LCD on the nucleo board. For the setup this time, I based the GPIO and SPI connections off of exactly what was connected on the board and based the setup off of the PCB setup I got working yesterday. The main change here from previous setups was making the clock prescaler 4 instead of 2 to let the SPI baud rate be 12 Mbps instead of 24 Mbps as it was previously because the setup I got working used the slower speed. For the code, I just used my code from yesterday which worked on the PCB and changed the LCD_NSS_PIN and PORT and the LCD_RESET_PIN and PORT to the currently wired pins and ports instead of the pins and ports on the PCB. I tried testing this and it didn't work. The code would run through all the tests and would write the values to memory correctly and read the values it just wrote correctly, but nothing showed up on the screen. Since this was the same code I got working on the PCB yesterday, I was convinced it had to be a wiring issue.

Remembering how I fixed this yesterday, my main theory was that the chip select or reset pins were always being set which was causing the screen to show nothing even though it had the data in its system to display. After looking through the wiring a few more times, I rewired it to exactly match the PCB pins and ports. This meant PA4 for chip select, PB1 for the reset and all of the other 18 pins being connected exactly as they are on the PCB. The only exception to this was the SPI_MOSI pin which had to use the backup PB5 instead of PA7 since the nucleo board doesn't have a flywire connector to PA7. I remade a new MX configuration with this new setup and tested it, but it still failed in the same way. It would go through the entire code, read everything that it wrote to any memory location correctly, the LCD was on since the power LED on the back was on, but nothing would display.

After comparing the code and setup to my working PCB one for a while, the only difference I could see was that the PCB SPI uses a prescaler of 4 to get a baud rate of 16 Mbps while the nucleo has a baud rate of 12 Mbps after a prescaler of 4. I tried reconfiguring this prescaler a few times, but a prescaler of 2 leading to a baud rate of 24 leads to the exact same issue and any other prescaler fails to read anything it writes so it fails at the reg_id check prior. After more discussion and debugging, Adhiksit and I instead decided to trust in the working PCB code and read up on the code engine and functions we had available to use to write code on the actual PCB once it was resoldered worked. Before that, we all also spent some time wiritng up the team ABET document. We split up the team section between the 4 of us and each wrote up our individual sections.

I spent the time before Juho finished soldering and continuity checking the PCB reading up on the official matrix orbital github for the EVE3 demo https://github.com/MatrixOrbital/EVE-Basic-Demo-STM32/tree/master on their setup and how they use the in built engine commands to display designs. I also used the LCD's programming manual to see all the available functions and commands which can be sent with some pseudo code examples. After Juho finished resoldering and testing the PCB and Alan ran his already working servo keypad code as well as his updated partially complete servo tempo code on the PCB, I tested the LCD code. It worked and displayed a blue screen just as it did yesterday.

With that done I started playing with the different LCD commands available from the matrix orbital library. I used the provided macros and the samples provided from the programming manual to try and replicate the examples they used. The initialization code has a test display list which gets sent and does successfully make changes on the LCD in the current version of the code so I first tried changing that display list to get different background colors and other basic displays. This worked pretty easily and I could change the screen color on the startup, but the command code I wrote yesterday still didn't work. The next roadblock was getting new display lists written. There are two methods to actually send commands to the LCD. The base method is to write to a display list at RAM_DL + an offset and to keep incrementing the offset. After the whole display list is written, writing a DL_FRAME to the reg DL_SWAP and writing a value to PCLK displays it on the LCD. The actual examples on the programming manual and the demo github only actually use this method for the intiial setup. For the rest, they use the command method where you write commands to RAM_CMD + an offset and then write to REG_CMD_WRITE to tell it how many instructions are in the command buffer which it then sends to RAM_DL and services the instructions. The benefit it that this allows for a wider range of use and another layer of abstraction. Since all the examples just use this method, I wasn't entirely sure how to just create a new display list.

My solution was to just manually set REG_PCLK back to 0 to hopefully refresh the system and then to write to RAM_DL + offset and send it as needed. This actually worked first try and displayed the screen color from the new command list after running through the code. I tried using the BEGIN command which allows you to start a bitmap, rectangle, line, etc. Once these are setup, you can set up vertices with the VERTEX2F command which sets up a vertex to 1/16 pixel precision or VERTEX2II to set up a vertex and a font for bitmaps. After learning more about these and testing them numerous times, I got the below code to make a background color, write different letters and draw lines of different lengths in different colors.

Entry 2: -----------------------------------------------------------------

Date: April 22nd
Start Time: 330pm
Duration: 5 hours

330-530, 830-1130 I started by discussing the USB UART IC issue with Alan. He had done a much deeper dive on the IC after I left last night and he said he thought he had figured out the error. He told me that the IC has two modes, the bus powered mode which is the default and the self powered mode which is what we currently have wired in our PCB. Apparently, the chip cannot power on since it has the wrong configuration so it can't actually identify a USB connection coming into it. The truly baffling part is that to reconfugre the chip, we need to power it on and use FT_PROG to write to its memory, but to power it on it needs to be wired in the bus configuration in the first place. It took a while for me to fully grasp this since it seems like such a bad design choice on their end, but essentially it looks like even though there are multiple configurations on the chip, you have to either use the default bus powered configuration or externally connect it to a breakout board with the bus powered setup to turn the chip on so that it can be read by FT_PROG and reconfigured to a self powered system.

After I fully understood his points here, us and Juho discussed our options going forward. The main options available to us were to either connect flywire the extra UART7 pins on our PCB to RX and TX of an external FT232RL device or to try a complex rewire of the PCB to attempt to switch from the current self powered configuration to the bus powered configuration. Juho said that flywiring the extra UART to the external device wouldn't be too difficult and that is also the device I prototyped with so the setup isn't too bad and the code is almost guaranteed to work. The main downside of it was that we'd either not get the PSDR for USB UART or we'd lose some integration points depending on the final situation. Alan had looked into this for a while and thought that the complex rewire from self to bus power could be possible. He showed us the PCB design and went through the exact wires that would need to be cut and the flywires that would need to be done to get it to work. it seemed like a pretty tight flywire so we decided to discuss it with the TAs first since Juho said that he'd definetely need help with a flywire that precise. We called over a TA and discussed it all and they said that they would discuss it with the rest of the TAs and look into the datasheet first since this would be a very invaisve process and could fry the PCB if we messed it up so they wanted to be sure this was absolutely necessary. We were almost sure that this was necessary, but it was hard to be completely sure that this would fix everything since the datasheet for the chip was pretty vague and had some minor errors so it was hard to trust it completely. I had to leave for a few hours while Alan and Juho continued discussing with the TAs and deciding on this.

After I got back from my office hours section, I started working a bit more on the LCD code. Adhiksit had sent me the current version of his code which wasn't working on the nucleo board so my goal was to try and get it working on the PCB. I looked through the code a few times and compared it with the demo code on the matrix orbital website https://github.com/MatrixOrbital/EVE-Basic-Demo-STM32/tree/master as well as the programming manual. I had looked through it a bit yesterday and the base functionality seemed to match what was asked of the programming manual for the initialization process. The code was also reading from every memory location that it wrote to and the values were being read correctly so it had to be with the GPIO pins or the memory locations themselves.

Based on the memory map, I looked through the code and checked each memory location that was being used. All the register values are offsets from RAM_REG and when looking through it I found a bug where he was writing to REG_DL_SWAP directly instead of RAM_REG_DL_SWAP. This was big since DL_SWAP is the register which tells the LCD that the data in the display list is ready to be parsed and acted on. I looked through the code again a few more times to be sure there was nothing else and found a bug in the read commands. The read address in the EVE3 communication needs to be prefaced by two 0's before the 22 bit address is sent which was missing in the current code. I added it by just anding the first byte being sent with 0x3f to 0 out the first two bits. Once I made these changes I ran the code and it didn't work. The code would go through the entire program, write the values to the memory locations, read them back correctly and all of it was happening through SPI so I'm not sure what was going wrong. I also thought it could just be the initialization commands so I tried using the command method the manual and github use to send commands, but it did nothing. This method involves sending commands to RAM_CMD instead of RAM_DL and then to send the number of instruction bytes left to read to the REG_CMD_WRITE register which will send the instructions and some special commands to RAM_DL and run the commands as needed.

After reading through it all a few more times, I was pretty confident in the code so I thoguht it was an error with the nucleo board or the wiring. Since we had the PCB set up, I tried running it on the PCB directly. I reconfigured the code with a new CUBE MX project which builds based on the microcontroller itself instead of the board. I ran the code and it did nothing on this end as well. I read through all the code, the documents and the setup again and then I found an issue. I was using the wrong reset and chip select GPIO pins in the code. This could mean that even though it was sending the data fine and the LCD had all the info it needed to display, reset was always pulled low so it could never actually display. I ran the code and it worked! The screen lit up and I could change the colors on it. The command code I added below didn't seem to do anything yet, but we had this working so I put that on the back burner and went back home for the day.

Entry 1: -----------------------------------------------------------------

Date: April 21st
Start Time: 8:30pm
Duration: 3 hours

I started by learning the PCB setup process from Juho and Alan and trying to get the USB UART code I had gotten working on the prototype on Friday working on the PCB. I made a new project with my code from Friday, but set up for the microcontroller instead of for the nucleo board like before and connected my laptop to the PCB, but my python code couldn't send the data over. I looked into it and it seems like my laptop doesn't recognize the connection to the PCB as a valid communication port and therefore pyserial can't send data through it. I thought that there could be an error with the wiring somewhere so I looked into the datasheet and saw an error in the connections. The TX of the microcontroller seemed to be connected to the TX of the chip and the RX to the RX. I remember making this mistake when setting it up on the prototype device so I caught it here, but I thought that maybe this worked here so I asked Juho about the identification problem and the TX-TX RX-RX connections. He looked into it and confirmed that it was an error and said that there was a chance this was causing the communication port error, but that he wasn't fully sure about that part of it. I was still a bit confused since the FT232RL, the device I prototyped with, was recognized as a communication port the second I plugged it in, even without power or any setup. After some research and discussion, we decided that it was probably configured differently and even though it has the same base chip, there is a full device there which may be dealing with this.

I spent some more time trying to debug the identifying the communication port error. The TX-TX and RX-RX most probably wasn't the issue here so I looked into ways to fix this. I asked a TA about it and they said that pyserial may not have perms to find certain unknown ports on my device so I should just use terminal directly. This made sense so I did some research into ways to find all communication ports through terminal or other means. I found the commands ls /dev/cu.* and ls /dev/tty.* to print out all serial communication channels. I tried both and they gave me the same info as pyserial did, my bluetooth and WLAN ports and nothing else. To sanity check, I tried to connect to the FT232RL which I know can be identified and ran both commands again and the device showed up on both of them so I know the commands were valid. There was also some info on mac forums on finding USB connections through system information so I tried those, but the PCB still didn't show up. I thought that it may just be MAC interfacing weird with the FTDI so I asked Alan to try and he couldn't detect the PCB and could detect the FT232RL, same as me. I discussed the issue with him a bit more and read the datasheet multiple times, but I couldn't figure anything out. He continued to look into it so for now I'll shift to trying to get the LCD working for a bit. I looked at the code Adhiksit had and talked about the errors with him for a bit. It seems like every write is happening correctly to the specific memory locations on the chip since the read at that same location returns the correct value, but the display still shows nothing. I read through the demo github and the manual for a bit, but I couldn't see any obvious mistakes in the code and my laptop was about to die so I headed out for the day.

=============== Week 14: =================

Entry 3: -----------------------------------------------------------------

Date: April 18th
Start Time: 8pm
Duration: 1.5 hours

I ran a few tests to confirm where I stood yesterday. It still looks like it can consistently get the correct num_drums, max_notes and num_notes per instrument as per the MIDI file, but it still gets a garbage value when I send 42 as a test send to make sure that the send and receive buffers are aligned properly. I started by adding breakpoints after every HAL_UART_Receive and by increasing the time.sleeps on the python side to 10s a piece so that I have ample time to analyse the values for each variable. After looking through each of them, it seemed to skip the start_time read since the value placed in the variable was garbage after the receive. Since all the receives on the int_buffer were working fine, I theorized that this could be something related to how doubles are dealt with in python vs C. All my doubles on the python side were of the type np.float64 where I packed them in struct.pack as "d" or as an 8 byte double. This should match what I am trying to receive on the microcontroller side which is an 8 byte value, but I was worried that np.float64 may be treated differently than a C double. I did some research and found that regular floats in python are the same as doubles in C so I made some test variables where one was a packed version of the np.float64 and the other was a packed version of the float. After testing, they both were exactly equal and more importantly, had a length of exactly 8 bytes which is what I was trying to receive on the C side so I am confused.

Since it was sending fine from the python side, I wanted to check if the value was actually being received on the microcontroller side. I added a status variable to check the return value of each HAL_UART_Receive call. Ideally these should all be HAL_OK, but my theory is that this read is just timing out. I ran the code and sure enough status returned HAL_TIMEOUT. I upped the timeout values to HAL_MAX_DELAY for each run of receive so that it would just infinite loop there instead of moving ahead with a garbage value and misaligning the buffer. Now that I confirmed it was being sent properly and the issue was that it wasn't being received, I thought that it had to be something with the double_buffer variable. As a long shot, I memset the double_buffer variable to 0 to initialise it at the top of the code. I hadn't done this for int_buffer and it still worked so I didn't have too much faith, but it couldn't hurt so I initialised them both. I ran it again and it ran through and seemed to get the start_time correctly. I removed the breakpoints after each receive and shortened the time.sleeps on the python side to 0.05s and it ran all the way through, not getting stuck at the breakpoint I placed at the test variable check for 42. I still wanted to be completely sure so I added a loop to go through and check each variable after receiving it and they all matched the data on the python side! Tested a bit more and it looks like it works on both ends. The code is below (some of the minor changes I made I talked about in previous entries but didn't include a full picture of them so this is everything).

Since the MIDI preprocessor to USB UART line is working, my plan is to first try and integrate Alan's servo code in with this and then the integration should be well along and we should be able to get at least 3 PSDRs by tomorrow (after testing on the PCB)!

Entry 3: -----------------------------------------------------------------

Date: April 17th
Start Time: 3pm
Duration: 4 hours

Adhiksit was having some weird issues with the LCD code so I came into lab to try and fix it. To start off with, I discussed how the EVE3 coding works with him and got some links with examples. After that, he walked me through the current version of the code before leaving for lunch. The current issue was that the code infinite loops at the point when it is trying to read until the special status register is 0x7C. All reads seem to be returning 0 even after all the base initialization. The base initialization involves sending host commands to set up the device and then looping until a read at a specific address returns 0x7C which indicates that the system is ready. For the host commands, the current code was HAL_SPI_Transmitting 3 bytes, the command, the parameter if there is one and lastly a dummy 0 byte. For the reads, the 22 byte address is prefaced by a 00 as per the document to reach 3 bytes for the address to send via HAL_SPI_Transmit. After this, one byte of data is read to get the read buffer which is used as the value read. The first thing I looked into was the host_command send protocol. I read through the programming manual and the datasheet and saw the general SPI protocol of 22 bit address prefaced by 10 for a write and 00 for a read being written for the address and then writing a dummy byte and then the data. They only mention the full protocol for reads and writes, but they do mention that the address should be prefaced with 01 for host_commands. This led me to believe that host_commands also need the 22 bit address to be sent and can't just send the command and parameters as it is doing now. I started searching for the specific memory address to send the host_commands to. I found the memory table shown below and saw the commands section of memory. I assume these are the ones for host_commands so I changed the code to send the address prefaced by a 01 and a dummy byte before sending the command id and the parameters. I tested this and the issue remained. It looks like the commands are being sent fine since it returns a HAL_OK, but I'm still not sure if it's actually doing anything since the reads always return 0.

After that, I started looking deeper into the read function. I found that it was missing the dummy byte so it was sending the address and that it wants to read in the first 3 bytes and then immediately tries reading. Looking through the docs again, it seems like the dummy byte can be either a read or a write so my theory is that the read is acting as the dummy byte read and then misses the actual read data being sent. To corroborate this, I added an extra dummy byte to the write, but the reads continue to be 0. I then looked into the example links to try and see what this code is doing differently. One of the examples was an EVE3 connecting to STM32CUBE through HAL and it was from the official creators of the LCD so I trusted that one. Looking through it, most things seem to be pretty similar to how the code is now. They do writes and reads in a similar way. One thing I found was that in their host_command section, they call attention to the fact that they do not send the address even though the manual says to, indicating that the manual was wrong. With this in mind, I moved the host_command code back to how it was before and kept all of the dummy write additions. With this new version, I tested it again and it still always reads a 0. I spent a while longer reading through this github and trying to see what it does in its initialization stage that we are missing, but I couldn't find anything. I was stumped here for a while so I decided to move over to USB UART.

I ran the code again and it seemed to reach the new_line send and check section of the code, but it errored out on the python side. I'm not entirely sure how to send a single char with struct.pack since I need to convert it to a bytes object and packing it with "c" didn't do that. It was also making it more annoying on the CUBE side since I needed extra variables and converters to extract the char. Instead, I just changed the code to send the int 42 and to extract and check on the CUBE side if the extracted variable is 42. This acts as a sanity check after every instrument to check if the sender and receiver are still aligned. I ran it a few more times and it seems to pretty consistently fail at the test check. At this point, I can see that sometimes the num_drums, max_notes and most of the time the num_notes values are all correct, but a lot of the time the num_notes values is just 0 and others the max_notes is just equal to the num_drums which means that something went wrong with that receive. I thought that there could be a chance that the receivers were just considering the previous value they got as having finished the receive for the current variable since I am reusing the int_buffer variable for all int receives so I added resets by memsetting int_buffer to 0 ater every run. My other theory was that the sender could be sending too fast which fills up the UART_Receive buffer and leads to values being dropped which misaligns it at test so I added some time.sleeps on the python side after every run so that the STM32CUBE code is always waiting at a HAL_UART_Receive. This ended up much better since it seems to always get the num_drums and max_notes correctly. It messed up num_notes on one run, but I couldn't replicate it so I just ignored that. After a bit more testing, I see that it is still getting a garbage value for the test variable, but at my breakpoint there I can see that the num_notes, max_notes and num_drums are all good so it must be something during the actual data sends. I'm done for today, but I will finish this tomorrow.

Entry 2: -----------------------------------------------------------------

Date: April 16th
Start Time: 2:00pm
Duration: 3.5 hours

I came into lab early to hopefully finish USB-UART today. Alan and Adhiksit were already in and were testing their servo keypad integration. Since we only had one board, we proceeded on a I test for a few minutes, then pass it to them and they test for a bit and then we trade back basis. After a few tests, I saw that the data being sent by the microcontroller wasn't being read by the python script so the python side would infinitely hang after the first run. On top of this, the microcontroller side seemed to sometimes somehow pass the second HAL_UART_Receive even though data is only being transmitted once. I double checked the sizes on both ends and on python the struct is being packed to send 4 bytes while on the microcontroller side it is HAL_UART_Receiving 4 bytes so I'm not too sure what's going on here. I added a timeout to the serial channel on the python side so that it could continue sending data instead of getting infinitely locked by the receive. This let it pass through the num_drums max_notes area, but it was still erroring out every time it reached a malloc. I tested it a few more times with breakpoints everywhere on the CUBE side and a high timeout on the python side so that it was slow and it looked like regardless of how much space was being malloced it would complain. One test, max_notes was 1 and it still errored out on the malloc(1 * 8). My main theory for this was that the heap memory was set up weird such that it wasn't allocated any space. I was doing some research into this when the team discovered a huge issue with the PCB.

We were working on figuring out how to actually boot our code onto the PCB since we didn't have a program header soldered onto it. Our original plan was to upload it through USB UART, but Alan did some research and found that the boot only works with UART1, 2 and 3. Our PCB only has UART7 and 9 set up so that was a big problem. We all started discussing potential ways forward. The boot worked with SPI1 so one idea was to buy a USB SPI1 header, to connect it to send the code over and then to disconnect it and use SPI1 for the LCD as planned. This would be pretty complicated and would add complexity, but it would be testable in a few days which was a big plus. We came up with some other ideas that wouldn't work and the absolute worst case was to change and reorder the PCB by EOD and get it by next Wednesday to hopefully get everything working in the next two days. We really didn't want this so when the TA's and professor came over we discussed it and they concluded that we could solder on some connections such that an STLINK could be connected which could potentially be done today. He discussed the specifics with Juho and he agreed that it was doable today.

After that was dealt with, I went back to fixing up my code. To start off with, I wanted to test out whether it would work with static arrays. I made a new project and wrote up some code to make the 2d array static 4 x 500 such that it can hold a max of 4 drums and 500 notes per drum. The code was fine, but I really didn't like the idea of using a static array here. I neede to set the size to a really large number such that no notes were cut which would just be wasteful for most songs and also wouldn't allow flexibility in the number of instruments. A lot of MIDI files have more or less than 4 drums in their drum set so this wouldn't allow for user choice on which drums they need and would waste even more space if the song has less drums. For an uploaded song with only 100 beats on 1 drum, this method would waste 22800 bytes and there's still a chance of a song with more than 500 beats dropping notes. I really didn't like this so I started looking deeper into working with 2d arrays in C to see if I was doing something wrong. After some research I found that I needed to first malloc space for the number of drums which I forgot to do. My code was just mallocing for notes_array[0], [1], ... as needed based on the number of arrays, but apparently in C you have to first malloc notes_array itself to tell you how many indexes you are allowed to access. I also did some research on the whole values from previous runs of the code corrupting the UART receive buffer. I thought that this may also be causing the code to escape from the HAL_UART_Receive without a value being sent. After some research into controlling the buffer through pyserial, I found the reset_input_buffer and reset_output_buffer methods which should clear the entire UART_Receive buffer and the pyserial receive buffer which should clear up the buffers between runs. Adhiksit and Alan were still working and they each also needed the microcontroller for their own tests so I left for today, I'll come back tomorrow to work on this.

Entry 1: -----------------------------------------------------------------

Date: April 15th
Start Time: 11am
Duration: 1 hour

Worked on writing up the new code to test. I started writing up all my new code so that I could test it next time I was in lab. I started by reading the docs to figure out how to receive data on the pyserial side. The docs had a .read() method on the serial class so I added it to the code. My goal was to get the CUBE side to echo everything it received back to the python side which would be stuck until it got it back which would allow it to send the next value. This would fix the timing issue from last time since there would always be at most one entry in the UART receive buffer. On the python side I added logic to read the same number of bytes it sent after every send, to store it into a variable and then added some prints of that variable and the variable.decode so that I could check if the value transferred correctly. On the CUBE side, I added HAL_UART_Transmits after every HAL_UART_Receive on the same int_buffer/double_buffer/char_buffer that was just received after I place it into a variable. In the end, the code looked like below:

=============== Week 13: =================

Entry 2: -----------------------------------------------------------------

Date: April 10th
Start Time: 5pm
Duration: 3.5 hours

Changed MIDI preprocessor to let it work as a 2d array on the microcontroller side, rewrote the UART code on the microcontroller side with this new format, tested and debugged The goal for today is to get the python and C sides working such that CUBE can store the values as a 2d array with variable rows for each instrument and columns for each entry. To start off with, I changed the dictionaries on the python side to each have as many entries as the most played instrument. The reason for this is that all this data is going to be transferred over to the microcontroller which needs to make them into a 2D array so it's easier to pad the entries which would be undefined with 0's in python than to receive only the specific amount of notes needed and then to leace the rest undefined. For example, if drum 1 has 200 notes (start time, velocity pairs) and drum 2 has 500, the microcontroller is going to have to malloc space for a 2*500 array either way so it's easier if drum 1's entries are padded with 300 0's so that we aren't reading any garbage values on the C side. Once I finished that, I tweaked the serial code a bit for this new format. This time I first send the number of drums followed by the max number of notes to allow the microcontroller to malloc space. Then I send the start time velocity pairs note by note followed by a new line character for testing. I do this for every drum and then close the channel when I'm done.

Next, I moved to the microcontroller and started rewriting the receive code. First off, the DTR checks and CTS sets and resets were making the code really clunky and seemed slightly unnecessary since all the examples in the github (https://github.com/STMicroelectronics/STM32CubeH7/blob/master/Projects/NUCLEO-H723ZG/Examples/UART) and in the example uses for the chip didn't do anything with it so I did some research on whether they are actually necessary. It doesn't seem like they are and they make it a lot harder to actually debug so for now I will write my code without it and I can change it if necessary. This simplified the code a good amount and led to it looking like the image below. The structure is just to HAL_UART_Receive a specific number of bytes and place it in a variable, while lining up each HAL_UART_Receive with a python CommPort.Send, both in the value being extracted and the number of bytes. I start by getting the number of drums and the max_notes so that I can loop through each drum to malloc a row in the 2d arrays. Then I loop through to get all the values to place in the arrays and free the arrays once I'm done.

After this I started testing and debugging. Testing this is pretty weird since the lab doesn't have any microUSB to USB C adaptors and the team only has 1 at the station. My laptop only has 2 USB C ports and the adaptor is being used to connect to the microcontroller. The chip needs a microUSB connected to it which it can convert to UART and I can use on the microcontroller. After searching through the lab for a while for a microUSB USBC cable, I couldn't find one so I was stuck. In the end I asked Alan if I could use his laptop for the python side of things since the lab does have USBB microUSB cables and his laptop has USB B ports and we set it up such that the microcontroller was connected to my laptop so I controlled the STM32CUBE side and the USB UART device was connected to his so he could run the python script to send over data. I set up the connections such that TX of the chip was connected to the UART RX of the microcontroller and started testing. The testing involved running the python script and the code at the same time and checking if it all goes through or where it errored out.

I ran into a weird driver error right off the bat, but from previous tests I know that usually means something went wrong with the CUBEMX setup so I made a new project, set it up with UART5 instead this time, copied the code over and tried building again. I was getting a lot of warnings/errors for incompatible pointer types. Looking into it a bit further, it looks like HAL_UART_Receive only outputs an array of uint8s so I can't directly map the array of 4 uint8s to the address of the ints even though the value should be the same (the 4 8 bits should make up the 32 bits of the int). I changed my code to make HAL_UART_Receive output a buffer of 1/4/8 uint8s and then I use some basic bitwise operators to loop through the buffer and shift the value into the char/int/double. The main idea here is just to default it to 0 and then bitwise OR the next 8 bits in the buffer at the correct location so the least significant 8 bits won't be shifted, the next least will be shifted 8 to the left and so on until all bits of the variable are filled. This worked fine for the ints, but I was running into an error with the doubles. I was getting an error saying that doubles cannot be shifted by ints or bitwise ORed so I did some research on how to perform bitwise operators on doubles. It turns out that doubles have some special in built formatting so bitwise operators on them aren't allowed by C even though it's technically correct here. I thought about what I could do here and in the end settled on just memcpy'ing the buffer at its starting location into the double which should get all the bytes into a double.

I tried a few more runs and was getting some malloc errors. I saw that I was mallocing the wrong amount of space at one point and at another the number of drums or the number of notes was getting a garbage value which made it attempt to malloc an impossible amount of space. I gave all the variables intial values and tried a few more runs, but it looked like a lot of values were stuck in the HAL_UART_Receive buffer from previous failed runs so I was reading values from where the code stopped last time instead of the new values I was trying to send. This was leading to more errors with the malloc and loops. Looking through all the variables at the fail points, it looks like it got the max_notes and num_drums values (72 and 2) at some point, but with the buffer misaligned at this point the tests weren't really valid anymore.

My other theory is that they just don't operate at the same speed so the sender was overflowing the buffer and constantly overwriting its own values with garbage. This seemed more likely and I see two main ways to fix it. One would be to slow the sender down by adding time.sleeps between every send to allow time for the microcontroller to get the value. The other was to add half duplex communication so that the microcontroller can send back the data it just received and the python code waits for the echoed value before resending or sending the next pair of values, emulating a basic stop and go scheme. This is definetely harder to implement, but it will make the communication a lot easier to walk through/ debug and will lead to much safer communication. For today though, I made some good progress and my laptop is about to die so I'll try again in a few days.

Entry 1: -----------------------------------------------------------------

Date: April 9th
Start Time: 1pm
Duration: 5 hour

This session I forgot to take pictures of a lot of intermediate steps of the python code so I'll show the final version and explain all the steps that came in between. To start off with, I realised that my current MIDI preprocessor file can only deal with 1 drum. I had the start_times and velocities arrays set as lists so if there were more than 1 instrument where is_drum was true, it would only take the last one's value. I took a pretty wide range of MIDI drum files and I definetely thought I heard more than 1 drum in some of them, but is_drum was only high in 1 instrument in all cases so it hadn't mattered yet, but I still wanted to account for it. To do this, I changed the start_times and velocities variables to dictionaries keyed by the instrument class and containing the normal array of notes/ velocities I've been using. I also noticed after a bit more testing that the notes arrays I've been printing have some duplicate notes. I can see multiple equal start times with equal velocity so I think they were added in by mistake. I added a check in my note loop to skip adding a new element to the arrays in the dictionaries if the start time of the current note is already in its dictionary array. This cut down the total number of notes from 234 to 144 for the Blues2.mid file which is a bigger slash than I expected.

After I had the preprocessor code in a good state, I moved on to looking into how to send the data from the laptop over USB. The main applications I had found were FT_Prog and PuTTy for making a virtual terminal and sending data over USB. These didn't seem too hard to use, but formatting the data from the python file and sending it over through copy and pasting would be really annoying. The alternative was to have python write to a txt file and then to send over the entire txt file through one of the applications and then to read it from the microcontroller. This didn't seem like a bad idea, but I wasn't completely sure how I could receive an entire file over UART. From previous examples and sample code I've looked through, HAL_UART_Receive takes the data into an array of uint8s so I'd need to make one giant buffer the size of the entire txt file and then parse through it byte by byte and set these to variables and use them as needed. It seemed like some extra work on the C coding side which isn't ideal, but more importantly I still wasn't sure about how to actually send the whole file through PuTTy or FT_Prog and on the end user side it would involve multiple applications and a lot of installations and guides to get it working. With this in mind, I pivoted to using the serial module in python (pyserial). From the user side, this would involve just running the preprocessor on a specific MIDI file while plugged into the device through the USB port and it would automatically process the file and send over the information in the correct format. I started by looking through the documentation for the pyserial module https://pyserial.readthedocs.io/en/latest/. The main takeaways here were to first pick the port to send information through and a baud rate to set up the communication channel, to open it if necessary, then to send data with .write and read with .read before finally closing the channel with .close after all data had been sent. I've gotten pretty familiar with the struct module from 50863 so I packed all values into doubles, ints or chars as needed. I'm not entirely sure if the chip flips endianess when converting from USB to UART so for now I'm just going to assume it doesn't as pack these normally. All the start times are packed as doubles since their type in python is a numpy.float64, the velocities are packed as ints since they are ints and between instruments I send a \n character to check that everything has lined up well on the microcontroller UART receive side. As for the port, I'm using /dev/ttyUSB0 for now since it's the default, but I'll change that when testing.

Once I had the code on the python side completed, I moved to the STM32CUBE code for receiving with UART. I started by following the structure I used in my previous entry where I set CTS, check for DTR, receive and then reset CTS for each data transfer. Below is the code I ended up with after the first attempt. The core idea here is that I would limit it to a max of two drums and wanted a max of 4 arrays, 1 for notes array and 1 for velocity array of each drum. Since I packed the data into structs of 1/4/8 bytes on the python side for chars, ints and doubles respectively, I assumed that I could just map the output of the HAL_UART_Receive to the pointer of a char, int or double and the function would memcpy it in correctly since it seemed to have some internal method of preserving the value (a value HAL_UART_Received into a variable shouldn't be corrupted when the function is called again based on the samples given on the github https://github.com/STMicroelectronics/STM32CubeH7/blob/master/Projects/NUCLEO-H723ZG/Examples/UART/UART_TwoBoards_ComPolling/Src/main.c). With this in mind, I first got the number of drums to check how many of these arrays were going to e used, then I would get the number of notes for a specific instrument and would loop through this number to receive all of the start times and would then loop through it again to receive all of the velocities for the instrument. The reason I split them instead of just merging the loops was to add more \n sends in between for testing, I will change it after I get this working. After receiving each piece of data I'd place it in the corresponding array and if there were more than two drums I'd ignore the rest.

After this the team showed up and we had our meeting. I was discussing the MIDI file preprocessing and data format with Alan since whatever I sent would have to be usable for the servo. The current plan was just to send time stamps which would tell the servos exactly when to swing using a timer from the start of the program and corresponding velocities for the force. This is a lot more exact than our normal method of beat editing since that needs a min time scale of 1/16 or 1/32 seconds for the beat editor to work while this could be far more precise since it just has the exact times a swing should occur with no rounding. This would mean that uploaded MIDI files would sound great, but they couldn't be edited with the beat editor since that would have to round any times not divisible with the min time scale. We decided that this would be acceptable since it's preferred over the user trying to edit an uploaded file and automatically messing up its timing even without making any significant changes. When I showed Alan my MIDI preprocessor results and tests and we discussed it, we found some weird contradictions. Alan had the program https://musescore.com which allows you to convert MIDI files into sheet music. I showed him my test cases and went through how each song had only one instrument categorized as a drum with the range of beats. When he put up the sheet music, we saw that the sheet music showed multiple drums on the same sheet. According to Alan and google, drum sheet music works differently than other instruments. Most instruments have a single sheet per instrument and notes referring to the timing and pitch to play it at, but for drums, all notes on a specific pitch refer to a part of the drum so all notes on the top line could mean the cymbal, all on the next the snare and all on the bottom the bass. This was a huge discovery since between when he found this out and when we first found the error I thought pretty_midi may have been broken so I was looking into mido and making a new preprocessor file. This also explained why the notes had duplicates, they were of a different pitch so they were referring to a different part of the drum and why each of the files only had 1 drum, each drum instrument class has a ton of drums hidden in them (blues2.mid had 4, just-drum had 8 and drum had 2).

With this discovery, I rewrote the code to account for this. I changed the dictionaries to be keyed by the different pitches instead of the instrument class and now each entry contains the list for that specific instrument. I also deleted the duplicate note skip code since it doesn't apply anymore and changed the num_drums and num_notes calculation since we consider different pitches as different instruments.

During the discussion, we also said that 4 drums should be our goal and even if we only do two, we still want to have all the information present on the microcontroller side so the user can potentially pick which ones they want to use. Due to this, I started rethinking my approach to the UART receive on the microcontroller side, opting for a 2d array where the row corresponds to the specific instrument and the column to the actual start time/ velocity. I have most of it planned out in my head so I'll come in tomorrow and finish it off.

=============== Week 12: =================

Entry 3: -----------------------------------------------------------------

Date: April 4th
Start Time: 5pm
Duration: 3 hours

Worked on preprocessing MIDI files into beat editor table data to be sent to the microcontroller through USB UART. My end goal is to take in a MIDI file and output all of the drum instrument's beats as an array. Ideally this should be at the smallest possible time scale so that the stretch PSDR for zooming/ tempo change will be easier to implement. To start off with, I researched on python modules for extracting information from MIDI files. The main ones I have found are midi, pretty_midi and mido. They all seem to have pretty good documentation with mido having https://www.twilio.com/en-us/blog/working-with-midi-data-in-python-using-mido as a good guide to get started as well as https://mido.readthedocs.io/en/stable/ for official documentation and pretty_midi having some good online examples like https://github.com/imalikshake/StyleNet/blob/master/midi_util.py and https://nbviewer.org/github/craffel/midi-ground-truth/blob/master/Statistics.ipynb. I also found the source code for all three in https://github.com/mido/mido for mido, https://github.com/vishnubob/python-midi for midi and https://github.com/craffel/pretty-midi for pretty_midi. Looking into it a bit more, they all could work for the purposes of this project, but I'm leaning towards pretty_midi. Python-midi or just midi has the most stars on the github, but it's also much older than the other two with the last update being 6 years ago so I'm not sure if it'll be compatible. Mido is older than pretty_midi, but it has been receiving updates as of a few months ago so that's not a big issue. The main reason I'm leaning towards pretty_midi is that the formus I've looked at online seem to sligtly favor it over mido. They both use their own class and are well documented so for now I'll design with pretty_midi and I'll keep mido as a backup if it fails.

During my research I also found music21 https://www.music21.org/music21docs/ which has extremely good documentation and seems to be the gold standard for parsing xml files. xml files are music files mainly used to transport digital sheet music as opposed to MIDI files which are more for beat and tempo information. MIDI is definetely more appropriate for the design, but I could add an xml mode at some point if needed since it also holds some beat information (although not as much as MIDI files). Before writing the code, I need some sample MIDI files to test with. I used https://bitmidi.com and https://mididrumfiles.com/2024/07/download_free_midi_drum_files/ to pick out some songs which I knew had drums. From bitmidi I downloaded drum.mid and just-drum.mid and from mididrumfiles I downloaded once of the jazz tracks for a smaller sized file for easier testing. I also downloaded a few other random MIDI files just to check that it is actually picking out the different instruments. After some initial setup I wrote some basic code to just print out all the different instruments. It seems like each instrument class has some in built information and the flag is_drum is one of them which makes differentiating easy. It picked out the drum in all the files with a drum and seemed to ignore the rest so this is a good starting point.

After messing with the module more I've found some more very interesting class functions which could be useful. For the general full file class functions I found get_tempo_changes, estimate_tempo and get_beats which I thought could be useful. get_tempo_changes returns an array of all the timestamps where the tempo changes and the corresponding change if there was one. In this file there was none so it's meaningless, but this could be useful for complex files for determining when it speeds up or slows down. estimate_tempo gives the average tempo of the entire song in bpm which could be useful for filtering out or slowing down files where the tempo is more than what our device can handle. get_beats says that it returns the time of each beat in seconds which seems like it would be exactly what we need here, but testing it doesn't line up with the other data so it seems like it's just a baseline set on file creation in cases where the author doesn't set it themself rather than an actual array of beats. As for the important instrument class properties, there are two big ones I found. get_piano_roll sais that it gives the piano roll matrix of the instrument after sampling every set amount of time. I thought I could use this to get whether or not there was a beat every so often or something to that degree, but the term piano roll seems to talk about pitch and I'm not entirely sure it applies here since the data just returned all 0's. The one which actually really helped was the notes attribute of the instrument class which returns a list of notes for the given instrument. After printing it out below, it looks like every note object holds a start time, an end time, a pitch and a velocity. All the data I printed out and the code at this point is shown below.

Now that I have the start time of every note and its velocity, I can extract these arrays for every drum and print it out. The three lists I got in the code below should be enough information for the microcontroller to receive and play the drums for. The main issue right now is just changing the data into a format where it can be easily sent over USB-UART to the microcontroller, which will involve converting to correct fp type and formatting it correctly, changing anything as needed to fit with the microcontroller's needs based on our beat table code on that end and potentially some integration with FT_PROG. I will also change it to deal with multiple drums when we need it, but for now I'll shift focus to the USB UART side of things over the weekend so that we can begin some integration.

Entry 2: -----------------------------------------------------------------

Date: April 2nd
Start Time: 2:30pm
Duration: 3 hours

Went into lab early, worked on USB UART, had team meeting. For this week I started working on getting the USB UART communication working so that we can upload a preprocessed MIDI file. I started by reading through the datasheet for the USB UART chip we are using. The important parts here were the pinout details and the CBUS configurations. The pinout had USB Data Plus and Minus as inputs, a UART TX and RX line as an output and input, CBUS configuration signals as inputs, an active low RESET signal, an RTS (request to send) output signal and a CTS (Clear to send) input signal. The USB signals will come from the USB port from a laptop, the RESET signal can be held high and the UART signals will be connected to the microcontroller so the main issue are the RTS and CTS signals and the CBUS configurations. CTS can be a GPIO output from the microcontroller and then RTS should be set by the chip and checked by the microcontroller as a GPIO input for the handshake.

For the CBUS signals, there are 4 signals which can be set in different combinations for a wide range of configuration options. I'm not entirely sure what configurations are necessary so I'll keep the defaults for now, but I will change these if necessary. Since these are inputs to the chip, I was looking into whether I would need to control these as GPIO outputs. These are being set by the host computer so it seems like the program FTPROG which the USB device needs to actually send data to this FTDI chip can set these config signals. I'm still not completely clear on this so I'll start looking into actually using this chip with a laptop.

During the team meeting, we went over our current progress and discussed the PCB. It seems like our PCB was delayed because we used 2oz copper which wasn't valid for our design. Alan and Juho went through the design with the TA's and professor Walter and then we resubmitted. As for the progress, we seem to be a bit behind. We have most parts individually, but we don't have much integration yet. We don't have a part to connect the LCD to the breadbord yet so the new LCD hasn't been tested. The servo keypad interactions haven't been tested on the breadboard yet even though they both work. We also don't have a complete USB UART transfer yet. We got a pretty good idea of our current progress so we'll need to speed up work this week for the servo keypad and USB UART testing and especially for next week once we get the PCB and can test the LCD.

After this I started looking through working with the FT230x chip. https://www.macrofab.com/blog/ft-prog-programming-ft230x/ was an extremely good link for the interface from the laptop side. From what it says, FT_PROG has its own terminal and GUI for setting all the CBUS signals and actually sending over the data in USB format. The links I found on the FTDI website seems to be for windows only so I'll test it on the lab computer. Now that the computer side is dealt with, the main part left is the interface between the chip and the microcontroller. I started by doing some research on the interface between the FT230xs and an STM32 chip. I found some STM forums where people asked this and the responses were weird. https://community.st.com/t5/stm32-mcus-embedded-software/hi-all-is-it-possible-to-communicate-between-a-stm32f429vgt6-in/m-p/243697 was the link where the other questions seem to link to. They're talking about interfacing between an STM32f4 and an FT230xs which is a different microcontroller, but should be similar to what I have to do. The solution on the STM forum seemed to be to implement a CDC protocol yourself and to contact FTDI for their exact interface details. The chip was added to reduce complexity of the conversion so it seems weird that the way to use the chip would be to just do the conversion yourself.

I spent some time diving down this rabbit hole and trying to see what the interface process would look like on the microcontroller side. I couldn't find any examples of someone interfacing between an STM micro and this chip, but after looking through the chip specs some more through https://components101.com/ics/ft230x-usb-to-uart-interface-ic and the datasheet, I came to the conclusion that it should just be a UART connection on the microcontroller side. I still don't fully understand why they were talking about CDC and virtual communication channels and implementing the protocol yourself on that forum post (mainly because the end solution was the person who asked the question getting access to the USB transfers list from FTDI and signing an NDA before implementing it themself). I will keep it in mind for if normal UART on the microcontroller doesn't work. For this, I started looking through USART implementation on STM32CUBE. https://github.com/STMicroelectronics/STM32CubeH7/blob/master/Projects/NUCLEO-H723ZG/Examples/UART/UART_TwoBoards_ComPolling/Src/main.c and https://deepbluembedded.com/stm32-usart-uart-tutorial/ were great resources for understanding how UART works on CUBE. The core seems to just be HAL_UART_Receive in this case since I won't need to UART transmit in this case. I made a basic new CUBE project and then wrote some test code. For testing I'll be using the bigger test chip instead of the FT230xs since the FT230xs is too small to connect on the breadboard and we don't have a PCB yet to actually test it. The test chip should have the same functionality. I made the connections with as a GPIO input and as a GPIO output and had my code check for DTS before receiving data and then flipping the CTS.

I'll test this basic code over the weekend once I get FTPROG working on the lab computer.

Entry 1: -----------------------------------------------------------------

Date: March 29th
Start Time: 11am
Duration: 3.5 hour

Worked on completing the legal and regulatory analysis and met up with the team after to discuss it and plan for the rest of the week. I started by looking over the slides again and reading through the two sample documents given. The slides had a good list of most generally important regulatory agencies to consider and while the samples both talked about pretty much the same agencies, they had great detail and structure. My first goal was to find out which agencies actually applied to this project. The acoustic drum machine uses a PCB with a microcontroller, multiple strong servo motors, an LCD display, wall power and regulators. I started by looking into the FCC since all the samples talked about it. https://transition.fcc.gov/Bureaus/Engineering_Technology/Documents/bulletins/oet62/oet62rev.pdf was a great resource and had a ton of information on the exact process behind becoming FCC certified and the situations in which it is necessary. Having electronic devices operating at a high frequency means that the FCC is a must since the device will be an unintentional radiator.

After I finished the writeup for the FCC, I looked into the situations where all the others would be needed. The IEC seemed to be optional in most situations and more for consumer trust and the CDRH, FAA and FDA don't apply here at all. The ROHS and CE are both necessary if the product is being sold in the EU so I decided to include them both since there isn't really a location barrier to the acoustic drum machine and drumming is around as popular in the EU as it is in the US. I looked through https://europa.eu/youreurope/business/product-requirements/labels-markings/ce-marking/index_en.htm and https://www.rohsguide.com/rohs-certification.htm for the exact details of both certifications and it seems like the system is more of an honor system with audits at a later date. Lastly, UL certification is usually optional, but looking through https://www.idimages.com/stock/what-products-require-ul-certification/ it seems like products directly plugged into the wall like ours require a certification so I looked through https://www.ul.com/resources/preparing-your-ul-mark-evaluation-us-and-canada for all the steps and wrote the section out. The patent side of this was a lot simpler since the team had done pretty extensive research on the patent and market analysis for the intial project proposal. I reused the three patents from back then since our design hadn't changed too much and they all still applied. The patents selected were the humanoid drummer robot, the drum session simulator and the practice drum set.

A few hours after finishing the doc, I met up with the team to discuss the paper and our plans going into this week. The main feedback for the paper were some minor grammatical errors and clarifications the team wanted. I added a bit more detail on the reasons for each regulatory agency being necessary and changed the citation format with the team and once everyone was happy with the paper I submitted it. After this we set our goals for the rest of the week. Juho was going to be working on soldering practice and prep for when the PCB arrives, Alan would finish servo testing, Adhiksit would work on LCD testing and I would help with LCD UI and work on getting the USB UART working.

=============== Week 10: =================

Entry 1: -----------------------------------------------------------------

Date: March 26th
Start Time: 2:30pm
Duration: 3 hours

I came into lab and continued work on the UI as well as on looking through all our parts to see if anything was missing. The shipment had arrived with all our parts so we opened them all up and looked through them. Most things were in order, but the LCD seemed to be missing the connector needed to actually test it on a breadboard. The connections for the LCD are too small for a normal wire to fit and we have the part to connect it directly to the PCB, but that will only come in a week so until then we can't test with it directly. We could order a flywire connector to allow us to connect it to the breadboard, but that would take a few days to arrive as well so we may just wait for the PCB to come in.

We also spent some time discussing the packaging for the final product. We have a general idea for how it all should look, but we're going to need to build it soon so we discussed the specifics. The three main parts for this product are the box, the stands and the drumstick holder. The box will hold the PCB and will have the LCD and keypad on it, the stand will be a solid base to raise the drumstick and not fall and the holder will clamp onto the stand and hold the drumstick. For the stand we can just use a regular music stand so as per the TA guidance we are looking into Headen hall. For the other two we are looking into the betchel innovation center. The box will just be a wooden box with holes in it for the wires so there isn't too much there, but the stand will most probably need some 3D printing. After all the team discussions I went back to looking into the LCD. So far all of the protoyping has been on a normal 2.2 in ILI9341 display, but the EVE that we just got uses a slightly different interface so I looked into that a bit more. The basics from a CUBE standpoint seem to stay the same, HAL_SPI_Transmit to send data through SPI and a CS pin that should be pulled up and down before and after sending data, but there are a few differences. Instead of a DC/RS pin to control the data/ command sending, this sends the command first, then the address and lastly the data where the command seems to be just 2 bits wide and the address will be 22 bits while the data can be as long as necessary. The start and stop is controlled by the CS pin.


The exact interface with CUBE seems a bit weird though since all of the examples in the datasheet seem to be using the BT81x controller with a bunch of in built registers and functions which I can't find the source code for.

Looking deeper, it seems like different registers exist on different points on the address space so to change a specific register I should be able to just HAL_SPI_Transmit to that address using the send format of 2 bit memory write command 01 followed by 22 bit address followed by data. The address space I found on the programming manual seems to contain memory for main memory (I'm assuming that's what's shown on the screen), font tables (I think that's for writing letters in pixel format), registers (the ones I found seem to be for setting up the parameters of the device like the pixel height and width and for the drawing color), commands and display lists which seem to be where all my display commands go and some other areas which I don't think I'll use.

I think I get the general idea of how this works so I'll write up some starter code for next time which I can test when the PCB arrives or we get the connector to connect it to the breadboard.

=============== Week 9: =================

Entry 1: -----------------------------------------------------------------

Date: March 12th
Start Time: 2:30pm
Duration: 3 hours

Now that I could control the LCD with SPI, I worked on cleaning up and modularizing my code to make progress on the UI. The code was pretty much all in main and had a ton of commented out and dead code so I first removed all of that. I first moved all of the initialization code to a function called LCD_Init. Once I had that, I started planning on the functions I would need to display the UI on the LCD. The UI is supposed to be a grid with text on the leftmost cell of each row indicating the instrument name, multiple cells corresponding to each time period and each cell being filled a specific color based on how hard we want the drumstick to swing at that time. I split this into 4 main functions, Write_Row to write a row of pixels a specific color to help create the grid, Write_Col to write a column of pixels to complete the grid, Fill_Box to fill a box of specific coordinates a specific color to fill the cells and lastly Write_Text to write words at a specific location to label the rows. Writing text is pretty horrible on pixel display based LCD screens since you need to map out every letter as a series of pixels corresponding to either the color or blank so I skipped that for now and focused on the other 3 functions.
Write_Row and Write_Col are pretty similar in how they work. First I need to set an address range and then I need to send the color data. For Write_Col the address range will need to be the entire column on the column side and only the specific row index on the row side while for Write_Row it's the opposite where it needs to entire row on the row side and only the specific column index on the column side. The main consideration with this was that the LCD we are going to be using has more than 256 rows and columns so it cannot be represented as one byte. For every row and column selection, we send two bytes to indicate the start/end row/col and we need to send these one byte at a time so we cannot assume the most significant byte will be 0. Instead, we need to split the 16 bit index into an index_0 representing the least significant byte and an index_1 representing the most significant byte. To do this, I just assign index_0 to index and index_1 to index >> 8. When converting to a uint8_t, it should automatically truncate the last 8 bits so this should work. The other consideration was for sending the color information. Each color in this format is 16 bits so I'm still not entirely sure if we need to send 2 bytes per pixel to represent the entire color or if we send half the color for one pixel and the other half for the next which makes it look like the correct color. For now I set it up as the latter, but I will test further if it doesn't work.

Next I moved on to the Fill_Box function. It has the same general idea and considerations of Write_Col and Write_Row, except that all 4 coordinates are used to determine the range instead of just 1 index for the column/row. I split each index into its individual bytes and then use them to send the row and column address information. After this was done, I send the color information as I did in the previous functions. For now it does only send 2 bytes per HAL_SPI_Transmit call which is pretty inefficient so I could change it to send more after I get this working.

After I finished all these functions I tried making a basic main function to test it. I did some research on colors that the ILI9341 accepts and found https://gist.github.com/Kongduino/36d152c81bbb1214a2128a2712ecdd18 which has a comprehensive list of hex values for each color. I picked 3 random colors, tomato, yellow and medium spring green in this case, and tried to write a vertical tomato line at index 20, a horizontal yellow line at line 100 and lastly fill a 50x50 pixel box from row and col index 30-80 medium spring green. I ran into an issue where I was trying to use the byte split of the total number of rows as a global variable, but C was complaining based on some weird type errors with going from a uint16 to a uint8 in a private global variable. I think it has to do with how CUBE is configured where global variables cannot be defined as an expression of another global variable so I moved this code into the functions and ran the code. The code ran, but the lines and the box weren't visible on the LCD. I added some test breakpoints, but I will test it further later.

=============== Week 8: =================

Entry 3: -----------------------------------------------------------------

Date: March 3rd
Start Time: 10pm
Duration: 2 hours

After the presentation prep, I got dinner and took a break before coming back to the lab to finish off the SPI interface with the LCD. I started by going over the LCD commands once again. Currently I was turning the display on with 0x29, setting the column address with 0x2A, the row address with 0x2B and then writing data with 0x2c. I found a few more commands which seemed like they may be useful to initialise the LCD. I added in 0x01 for a soft reset in case it started in a weird state, 0x11 to turn sleep mode off in case sleep mode was on even though my display was on and 0x13 to turn normal display mode on. I looked through the rest of the commands in https://docs.mikroe.com/mikrosdk/ref-manual/group__ili9341__commands.html, but I didn't find any other which seemed like they were needed to initialise the LCD. I tested the code and once again it ran, but the LCD still only showed a white screen from the LED backlight. I did some more research on these initialization commands and saw that you sometimes need to add delays for them to fully process so I added 100 ms delays after each initialization command using HAL_Delay and ran the code again, but the issue remained.

After this I spent a bit of time going over the githubs and code bases I had found again to see if there was anything I was still missing. I had gone through them all quite a lot at this point so I had identified that the useful links were https://www.micropeta.com/video37 and https://www.digikey.com/en/maker/projects/getting-started-with-stm32-how-to-use-spi/09eab3dfe74c4d0391aaaa99b0a8ee17 since they were the only ones which actually used the HAL commands and CUBE. I looked through them both a few times and couldn't find anything I was missing. I tried changing the addresss ranges, the chip select, the reset and the number of bytes I sent at once with HAL_SPI_Transmit, but every time it just showed a blank white screen. I even tried varying the color buffer since I thought that maybe the color I was using wasn't acceptable so I tried 0xFFFF and 0x0000 for white or black and 0xF8F8 for red since I knew these were allowed. Still the same error.

I had a decent amount of confidence in my code at this point so I moved on to triple checking the wiring. I disconnected everything and started rewiring it wire by wire while checking the flywire mapping and the CUBE MX mapping at each stage. While wiring the MOSI pin, I found it. On CUBE MX, when you enable SPI1, it automatically sets MOSI to PA7. On the datasheet for the board, it says that PA7 is internally connected to D11 which is PB5. I trusted this and had been wiring to PB5 since there is no mapping on the flywires to PA7 and it clearly says that "PA7 is used as D11", but this time I changed the setup on CUBE MX to make PB5 the SPI1 MOSI pin and wired MOSI to PB5. When I ran it, it worked! I was able to see a blue section on the LCD. I updated my prototyping progress slides and headed back home.

Entry 2: -----------------------------------------------------------------

Date: March 3rd
Start Time: 2:30pm
Duration: 4 hours

I started by running yesterday's code again and looking through what happens at each breakpoint. It seemed to be going through the code fine and all the HAL_SPI_Transmit calls were returning HAL_OKs so I was unsure what the issue was. I went through the wiring again and tried wiring the pins I had left unconnected. I based it off of the wiki for the LCD so I connected the LED pin to 3.3V, CS to ground and RESET to 3.3V. I testing my code again and it gave the same error as before where it ran through the code fine and returned a HAL_OK on every transmission, but the LCD only showed a white screen this time. The white screen seemed to just be because the backlight was on which I double checked by tying it to ground instead which led to the screen being empty again.
I did some more research and found some confliting information on whether the RESET pin was low active or high active so I rebuilt the project and rewired it with RESET and CS being connected to GPIOB8 and GPIOB6 respectively. I tested all combinations of RESET and CS, but they all led to the same issue of a white screen.



I had been working on this for a bit and was getting stuck so I switched to working on the presentation for a change of pace. I put together my slides from the template, the examples and all of the previously submitted reports and then met up with the team to discuss. We mainly just went through the slides, identified all the missing parts and then did a general walkthrough of the entire presentation. The main things we identified were the block diagram missing some parts, the prototyping progress missing, the PCB layout slides not being completed yet and some information missing on the timeline and mechanical overview. The timeline didn't have multiple people assigned to the same task yet, the mechanical overview was missing information about the stand and the block diagram was missing the level shifter and bus widths. After we discussed changes to be made and updated the slides, we did a run through of the presentation where we went through every slide and the presenter would say their key points and try to present it. We also had everyone specify what all extra information they would add before the presentation and how long they would take. We weighted the presentation time more heavily towards the PCB and electrical schematic sections and made sure we would present for ~45 mins.

Entry 2: -----------------------------------------------------------------

Date: March 2nd
Start Time: 5:30pm
Duration: 3 hours

After last session, I realised that there was most probably something wrong with my interface with the driver. I started by double checking all my wiring and the MOSI, SCK, CS, 3V3 and GND connections were all wired so I looked at the labelling to the right of the flywire ports closely to try and see if they were all correct. Each label to the right of the ports had 2 seperate labels so I still wasn't sure which coresponded to the right side and which to the left. Alan was also in the lab so I discussed it with him and he said that they were both probably referring to the same port on the right so MOSI D11 just meant that the right port was D11, not that MOSI was the left and D11 was the right. To corroborate this I looked back over the nucleo board document link here. I started by skimming through it again and then tried searching for specific pins to see how they mapped. For example, I wanted to see if D11 was assigned to MOSI anywhere. While doing this, I made a huge discovery. I found a mapping of flywire ports to microcontroller GPIOs. Not having this was the biggest roadblock to wiring since a majority of them weren't labelled on the board and the ones that were labelled were done so in a really small font with labels bleeding together. The reason it slipped past me the last few times I read through the document was because the mapping was under the extension connections section for some reason so I assumed it was an external connection originally. I spent the next bit matching every pin I knew on the board with the pins on the CUBE MX image and the image I found. After seeing that they all matched, I finally had some confidence in my wiring.

After I checked the wiring for the MOSI, SCK, CS, 3V3 and GND connections, I started looking deeper into the unused pins to see if I was supposed to be using any of them. I currently had the pins LED, DC/RS and RESET left floating while CS was connected to ground. After doing a bit more research into these I found that DC/RS indicates whether the data being sent was a command or a data, where it has to be 0 for commands and 1 for data. I had just been sending data all this time so I had tried tying it to ground at one point, but now I realised that this driver requires certain commands before data can be sent. I went back to CUBE MX and picked an unused GPIO pin (I picked GPIOB 7 since B6 was being used for CS) to assign as a GPIO output. I also changed the SPI settings to make the SPI mode master transmit only instead of master full duplex because I'm not actually receiving any information from the LCD so there's no point of it. I did some more research to find these commands and found this link: https://docs.mikroe.com/mikrosdk/ref-manual/group__ili9341__commands.html which has all the commands as the hex value that needs to be sent for it. What stood out to me were the commands 0x29 for turning the display on and 0x2C for writing to device memory. I moved to the code and wrote up a quick function to send a command and another to send data. Data and commands are both sent 8 bits (1 byte) at a time and the only difference is whether the DC/RS pin is set or reset so these are both quick 2 line functions to set/reset the GPIO for DC/RS with HAL_GPIO_Write and then to send the byte with HAL_SPI_Transmit. Once I had these functions set up I tried to send the display on command, send the memory write command and then send some arbitrary color data. I based it off of https://www.digikey.com/en/maker/projects/getting-started-with-stm32-how-to-use-spi/09eab3dfe74c4d0391aaaa99b0a8ee17, but I used the commands from the wiki since the link was using SPI, but not this specific LCD driver. I also used https://www.micropeta.com/video37 for a better example of interfaing with the ILI9341 driver. I ran the code and it did nothing. It ran till the end and didn't error out anywhere, but nothing was displayed on the LCD. I did some more research and found that I needed to also assign a column value and page value through the commands 0x2A and 0x2B. The format looks to be a range where you need to send 16 bytes to indicate the start and then another 16 for the end for each of 0x2A and 0x2B. I just sent the range 0-100 for each since the display is 240x320 so that I could see a visible amount of the screen colored. I ran this and it still didn't display anything so I left to eat dinner.

Entry 1: -----------------------------------------------------------------

Date: March 1st
Start Time: 6pm
Duration: 1 hour

Had a team meeting to discuss the midterm review presentation. Mainly focused on dividing up slides, looking through the examples and generally discussing what we should be talking about for each slide. In the end we divided up the work and I had the block diagram, packaging design, prototyping progress and software overview. I looked into each of these sections on the examples and the template and made a general plan. I hadn't made the slides yet, but I had most of my talking points thought out and knew most of the images I needed to add when I would make it tomorrow.

=============== Week 7: =================

Entry 3: -----------------------------------------------------------------

Date: February 28th
Start Time: 6pm
Duration: 3 hours

Worked on debugging the LCD interface with the microcontroller. I started with the circuit and code from Wednesday. I tried rerunning it while holding the wires in the loose holes, but I was running into the same error where the rxbuffer was empty, meaning that no information was being received. I added a lot of LED checks along with the breakpoints throughout the code for debugging and the correct LEDs were turning on (flashing red when it hit the error handler and yellow when I made it skip the error handler and go to the final infinite loop) so it seems like my code was interfacing with the microcontroller, but nothing showed up on the screen. I thought that it could be because the slave select signal was connected to a GPIO which I wasn't setting so I tried setting it to ground so that the LCD would always be selected, but it didn't change the issue. After testing a bit more I decided to use the flywire holes at the top instead. The main issue with wiring here was that there are no real labels. Even the data sheet only shows the general blocks like below.

The only pins I really needed to wire were GND, 3.3V, MOSI, MISO, SCK and SS so I focused on trying to find labels for those. Each row on the SPI side has 2 labels, but it's really hard to see and unclear which is for the right and which is for the left. After finding the rows for SCK, MISO and MOSI on the microcontroller, I just assumed that the one written first was the left hole and the other was the right hole. I plugged in SS to PB6 as I had initialized in my setup, plugged in GND and power and then tested again. The same error persisted so I tried every combination of SS being GND/PB6 and one written first being the left or right hole, but they all gave the same error. The wiring was much harder to set up since most jumpers weren't working, but the connections felt stable this time. I went back to the datasheet for the prototype LCD and compared it to the actual part which hasn't arrived yet and the other prototype we had available. The LCD had LCD ports which didn't match anything on them and SD ports which followed SPI so I had been using the SD ports, but I thought that maybe the SD card interface was different so I switched to the second LCD. I rewired it and tested every combination like I did with this one, but the error still persisted. The wiring was simpler this time since this LCD had fewer ports and was more in line with what we will actually be using, but it still threw the same error.

After all of this didn't work I was left doubting both my circuitry due to the bad labels and my code due to the consistent failures. Since I had spent a long time messing with the circuitry, I switched focus to messing with the code. Most of my research thus far had been on setting up SPI in CUBE since I couldn't really find anything on the specific LCDs, but this time I focused on CUBE interface with any LCD instead to see if maybe I was missing something. I found an STM forum post asking about this which led me to the official STMCUBE starter course. I downloaded the slides and went through them to see if there was anything about the platform I was missing. The general idea seemed to be the same as what I knew, start a project based on your board, turn on the necessary features, set pins as inputs, outputs or a specific value, generate the starter code and then only write within the comments saying user code here. There were some good examples, but most were from the github which I had already gone through and none were about interfacing with an external device.

I then moved to research on more specific examples. I found a good link on the general setup and initialization of an LCD. It didn't seem to even use the HAL SPI function and instead just treated the pins as GPIOs. I ignored this last time since I thought that the HAL SPI function was a simpler way to achieve the same purpose, but this seemed to fit the ILI3941 driver specifications better so I had a new direction to go in. https://e2e.ti.com/support/microcontrollers/arm-based-microcontrollers-group/arm-based-microcontrollers/f/arm-based-microcontrollers-forum/471799/initializing-ili9341-based-color-tft-with-tm4c123-using-spi-gpio

After a bit more research I found another forum post which led me to the official CUBE friendly github for the driver out LCD uses, the ILI9341. I couldn't entirely understand what the code was doing and it seemed like most of the actual functionality was happening in the backend so I kept it in the back of my mind as a backup in case all the alternatives fail. I then looked through a few youtube videos on interfacing with an LCD using CUBE and although there wasn't too much information on the setup process, I found one of the githubs to be pretty useful. It also followed the use HAL GPIO functions instead of HAL SPI functions to do SPI method and based on the video it definetely can work so I'm going to try building off of this and see where it takes me.
https://github.com/4ilo/HD44780-Stm32HAL/tree/master - Link to github with seemingly useful LCD code
https://github.com/STMicroelectronics/stm32-ili9341 - Link to official ILI9341 driver github
https://community.st.com/t5/stm32-mcus-products/tft-screen-library-in-stm32cubeide/td-p/638275 - Link to forum with similar question

Entry 2: -----------------------------------------------------------------

Date: February 26th
Start Time: 2:00pm
Duration: 4 hours

Worked on trying to get something flashing on the LCD using the microcontroller. I had a majority of the code completed so I first tried to finish off the rest of the code. I based it off of both the SPI interrupt example and the SPI polling examples. I didn't have anything to use as the interrupt so I mainly focused on the polling example. The intiialzation and setup was already completed for the SPI interrupts so I can add it in at any time as needed, but I wanted to start off by getting something on the LCD so I ignored that part of it for now. The main difference between the two examples was that the interrupt example uses a signal caled wtransferstate which can be in the wait, complete or error state. This variable allows the program to continue based on its state and is controlled by the interrupt handlers. For polling we just wait until the spi transmit and receive is complete before checking if it was done correctly. Wrote the below code to write to the LCD. The main function used here is HAL_SPI_TransmitReceive which actually does the data transfer. It takes in a TX buffer, tries to write it to the receiver and stores everything it manages to write as an RX buffer. It outputs either a HAL_OK if everything went well, a HAL_TIMEOUT if it took too long or a HAL_ERROR if something else went wrong. If we get a HAL_OK, I compare the tx buffer and the rx buffer to make sure that all data was transferred. I tried seeing if there was an in built function to compare buffers, but I couldn't find one and the example seemed to write their own so I just wrote a simple one. I set the transmit buffer to an array of uint8's containing a few 0xFFs to display some white pixels and some other random hex numbers to show random splotches of color on the screen. After tweaking the header files a bit, I had the code done so I came into lab and moved to the breadboard to setup the circuit.


Both the dev board and the LCD were very weirdly designed which made setting up the circuit way harder than expected. The microcontroller has two sets of connections, the holes at the sides and the black holes on the front connectable through flywires. The side holes are labelled well on the board and on the datasheet, but are very loose and need to be soldered before use. The flywire connections are easy to use, but their labelling is near impossible to read on the board itself and doesn't exist on the datasheet.
B
Since I couldn't find any labels, my first attempt was to wire using the side holes. I didn't want to solder them yet so I just tried placing flywires in and holding it so that the wires could touch the edges. The other issue came with the LCD. The labels for the LCD are all on the bottom so they would all be invisible after it was plugged into the breadboard. I tried marking the pins on the top so that I could remember which was where and then I tried plugging it into the breadboard. The LCD was of a weird size so both sides couldn't actually fit in the breadboard. The left side fit pretty easily, but the right side couldn't fit unless I bent every single pin on that side a substantial amount which would be even harder in this case because the pins were really hard to bend. I decided to just use flywires to take in each pin and then connect them directly to the microcontroller.

After I had this wiring set up I tried actually running the code. I ran into an issue where the built didn't complete. I looked into it and for some reason the folder wasn't defined as a STM32IDE project and was defined as a normal folder instead. This was really weird since I set it up the same way as I did my previous version and that one was defined as a project. I looked into this for a bit and fixed the error by finding the .ioc file, setting up a new project with that file and then readding in all of my code. The .ioc file contains all of the setup configurations so after replacing all of the code I had a version which was defined as a project. This allowed me to build which worked and compiled. There were some syntax errors and typos, but after some quick fixes I had it running. I ran the code and it seemed to be doing nothing. I added some breakpoints to track the code and had the different LEDs flash on and off depending on what was happening. I had it flash red on error, yellow if it timed out and green if it worked. After some testing I found that it did reach the HAL_OK case, but the rx and tx buffers were unequal. I added another breakpoint in the buffer comparison loop and found that the rx buffer was empty so it wasn't actually transferring any information. I assumed that this was because the wiring wasn't exactly stable since it came out whenever I wasn't actively holding the flywires and so there's a chance it couldn't transfer any information. I decided to end for today and try again with the flywire holes on the microcontroller on Friday.

Entry 1: -----------------------------------------------------------------

Date: February 22nd
Start Time: 6pm
Duration: 1 hour

Had a team meeting to discuss the software formalization. Adhiksit had finished a majority of the doc so we mainly just discussed all the aspects of the software. The main points of discussion were the MIDI file preprocessing and usage as well as the updated state machine. For the MIDI files, Adhiksit wanted a specific application that we were going to use to add that to the licenses so we looked through some options for a bit and chose in the end. We also went through the general process of passing the MIDI file into the application, trimming out all non drum information, passing it through a python script to trim out any more unnecessary information and then sending it over to the microcontroller through USB where it loads it in, reads the file and stores its information as a beat editor table with the necessary beats at each time stamp. The main change with the state machine was that the IDLE state was removed and integrated with the EDITOR state and functionality to go from the UPLOAD state to the EDITOR state was added. The main reason for this is that previously we decided to not include the functionality to go from UPLOAD to EDITOR since we deemed it too much extra work and the user could just edit the MIDI file on their own device before upload, but we realized that to play the MIDI file it would need to be converted into the beat editor format anyway which is all that's necessary to go to the editor state so the state machine could be reorganized to account for this.

=============== Week 6: =================

Entry 4: -----------------------------------------------------------------

Date: February 21st
Start Time: 6:15pm
Duration: 3.5 hours

Came into lab and worked on interfacing with the sample LCD given. I started off by once again comparing the started code generated by CUBE after my setup on Wednesday and the github I looked through yesterday to get a better idea on what all I would still need to add. Since I'm planning on using SPI to interface with the LCD, I enabled the HAL module for it and then started writing some code to enable the SPI as in the example, but I was running into an undefined file error where it couldn't find the spi config file. Looking at the git tree, it doesn't seem like they made it anywhere in the example folder so I assumed that there was something wrong with how I created the starter code. I believe that because I just generated the code without setting anything up in the CUBE MX app, the firmware needed for spi wasn't in the code.

Considering this, I tried creating a fresh project in CUBE. My first idea was to generate one using the example generator option in MX. The application allows you to create a new project from a specific microcontroller, a specific board or a specific example. Previously I just did it from the NUCLEO-H723ZG, but I thought that if I could generate from an example with SPI interfacing to an LCD, I could use that for general testing. After looking through it for a bit, I couldn't find any in built examples which had the correct dev board and SPI interfacing with an LCD. Instead, I narrowed my search down to just examples with SPI interfacing and the correct microcontroller and found a few (presumably the same ones from the github which is what I wanted to begin with). I installed the firmware and then tried going to the next stage where I would select which all functionality is being utilized, which interrupts are being used, which pins are doing what and generate the starter code, but it wouldn't let me. After messing around with it for a while and researching CUBE, I realised that these examples aren't compatible with the MX application so even though they are listed since they're on the github, they can't be used as a base to build off of. All the firmware downloads also ran me out of space on my laptop so I decided to just build off of the board for now instead of from a different example

I did some more research and found a really good walkthrough for setting up SPI on CUBE. I followed the steps in https://www.digikey.com/en/maker/projects/getting-started-with-stm32-how-to-use-spi/09eab3dfe74c4d0391aaaa99b0a8ee17 to setup the HAL layer after selecting the board to build off of. First I selected SPI as a necessary feature, then I changed the GPIO pins as required to configure the slave select bus, then I changed the clock prescaler to get a good baud rate and finally I enabled global interrupts and changed the data size to 8 bits to get all the setup to generate correct starter code. Doing this seemed to work since after adding in the SPI configurations and enabling it in the HAL layer, the code no longer threw an error for the SPI config file like it did last time.

Now that I had some basic SPI code, I moved my attention to trying to breadboard the connections between the microcontroller and the LCD. I knew from the HAL layer that PA7 was MOSI and PA5 was SCK for the microcontroller and since MISO isn't really required here that should be enough on the master side. The only issue with this is that it's really hard to find these pins on the dev board. I even found a labelled layout on the user manual, but it doesn't specify the exact pin numbers except in really small mashed together text on the board so my current plan is to just show it to all members of the team before plugging stuff in so that I don't misconnect it. The LCD on the other hand had way more pins than I expected and I struggled to find which ones were relevant here. I assumed that the driver would follow a similar naming scheme and structure to all the ones I researched online since they're all labelled ILI9341, but this didn't have an explicit MOSI and MISO signal and also had a bunch of extra signals I couldn't figure out the use for. The signals seemed to be split into LCD_ signals and SD_ signals where LCD has a RST, CS, RS, WS, RD and D0-7 which SD has an SS, D0-1 and an SCK. After some research I found that SD_D0 just means MISO and SD_D1 is MOSI so I'm planning on just ignoring all LCD_ signals for now and seeing what happens. With all this out of the way I decided to stop for today and continue the interfacing tomorrow evening. I have a good chunk of the code so I just need to finish off the code, be careful while connecting to the microcontroller and make sure to remember to give the LCD 5V for power.

Entry 3: -----------------------------------------------------------------

Date: February 20th
Start Time: 9pm
Duration: 2 hours

https://www.youtube.com/watch?v=dnfuNT1dPiM - I watched the first part of this youtube video to get a bit better acquainted with CUBE. I mainly just looked over what I skipped while doing the setup yesterday (the HAL layer setup stuff which allows you to custom configure all features and pins on the dev board before you generate the starter code) and the worktree to understand the code. After looking through this for a bit, I moved to the official github to look over some examples.
https://github.com/STMicroelectronics/STM32CubeH7/tree/master - Adhiksit showed me this official github for the STM32H7 series so I spent the day going through some examples and trying to understand what's going on with this and how the code is structured. The official STM32CUBEH7 github is pretty massive and has a ton of code and files I do not understand. There are multiple examples for each dev board so I focused on the given samples for the NUCLEO-H723ZG. There are examples for most use case interfaces from BSP to DMA to RCC setup so I mainly focused on the GPIO section to try and understand regular GPIO usage and the SPI section to get a better idea of interfacing with the LCD. Under SPI I mainly focused on the SPI_FullDuplex_ComDMA and the SPI_FullDuplex_ComIT for info on both direct memory access and interrupt based SPI to compare them and see the implementation differences.
Each example has a main .h and .c, a hardware abstraction layer config header file, an interrupt config .h and .c file and a system setup file. From what I understand, the HAL layer config file just lists out all features of the microcontroller being used in the program and interface with the driver to enable the required components. The it(interrupt) files just set up all of the different enabled interrupt handler functions. System sets up all of the clock configurations (RCC) as well as the startup and shutdown functionality. These files were more or less the same through all the examples I checked with the main differences being SPI not being enabled in the HAL layer for the GPIO example and different interrupt functions. Most of these were also autogenerated in the project I made day yesterday save for the interrupt files and handlers. The main files which changed dramatically between examples were the mains and the HAL MSP file. The mains just initialised the HAL layer, the MPU, the CPU Cache, the clocks and any other modules necessary (SPI, BSP, etc) and then either did any necessary GPIO configs and then infinite looped. I still don't entirely get what's going on in the HAL MSP file. It only exists for the SPI examples and seems to take in some sort of SPI handler type and then enable and configure all necessary clocks, GPIOs, interrupts and DMA for the DMA example. I'll look into it more in lab while coding tomorrow.

Entry 2: -----------------------------------------------------------------

Date: February 19th
Start Time: 3pm
Duration: 2.5 hours

Worked on getting the IDE set up on the lab computer and on my personal one, met up with the team and the professors and TAs and discussed options for a new stretch PSDR. In the meeting the professor told us that he believed our second stretch PSDR (Adding in 4 drums instead of 2) wasn't enough design effort for a stretch PSDR since we would mainly be reusing code from our main functionality. We spent some time discussing potential stretch PSDRs. The first idea was to bring back our old idea of adding in rotary actuators to hit either the edges of the drum or the center. The main issues we came up with for it were that we'd need an extra motor for each drum which would increase current and power by a lot so we'd need potentially dangerous amounts of current to account for worst case current use and that MIDI files don't contain information on where to hit the drum so this is mainly intuitive and would be very difficult to account for and not too useful in a learning context focused on timing. Our device has a ton of features so we had a lot of other options for stretch PSDRs. We discussed options such as adding saving functionality so that people could save and potentially export edited music, but in the end we went with designing the converter to get power to all components. We went with this because Juho said that he was already working on this in the background and currently it wasn't in our description anywhere so the professor said they would assume we used prebuilt modules. This is definetely a doable stretch PSDR so it should be a good bonus.
After the meeting and disussion, I resumed work on setting up the STM32CUBE IDE on the lab computer. I ran into an issue where the system required admin credentials to install new applications on the device so I tried using system workbench instead. I did some research on the differences and the consensus online seemed to be that system workbench was inferior in every way. I considered just moving onto my personal computer, but I didn't have a USB MINI to USB C cable on me so I chose and the CUBE setup link said that a system with windows 7 or higher was required so I took that to mean that it wouldn't work on mac. I tried to setup the board on system workbench, but the application was on an old version which didn't have the dev board or microcontroller that we were using for the project. I tried updating it, but after 10 minutes of loading and scanning for updates it said that it also requires admin credentials. I discussed this with the TAs and they said that we there was an admin account that we were allowed to use, but while doing research on my laptop I saw that CUBE and all of the other required applications actually had mac versions so I think the STM wiki installation page just hasn't been updated in a while. The TAs also said that it would be faster on my laptop so I just set it all up on my laptop. To do this I just followed this link: https://wiki.st.com/stm32mcu/wiki/STM32StepByStep:Step1_Tools_installation#Install_STM32CubeIDE to download the CUBE IDE, the CUBE MX for starter code and the CUBE programmer for the coding and then setting up a new project.
I then used CUBE MX to generate some basic starter code to do some testing as shown below. All I really needed to do was enter the dev board and it took me to a page where I could configure each pin on the microcontroller as well as all sorts of functionality. For now I just left all of that as the defaults and generated the starter code before going home.

Entry 1: -----------------------------------------------------------------

Date: February 15th
Start Time: 6pm
Duration: 1 hour

Had a team meeting to discuss the mechanical overview and finish off the bill of materials. Juho had finished the doc so the rest of us mainly just went through it, gave feedback and made sure we all understood the mechanical design. The general design was similar to what we had discussed previously with a wooden box for the main design containing the PCB with the keypad and LCD on the outside with wires connecting to the motors which each have their own stand and holder. The main additions were the extra considerations for a potential cooling fan and the specific materials. For the fan, the box adds airflow holes and some space in the box and for the materials we have wood for the box, 3D print material for the motor packaging and either metal or more 3D print material for the drumstick stand. For the bill of materials we just went through every part we decided on and googled the price and device info. Found most of them on amazon or digikey along with the manufacture info except the motor and microcontroller which were supplied and manufactured by their parent company.

=============== Week 5: =================

Entry 3: -----------------------------------------------------------------

Date: February 14th
Start Time: 12:30pm
Duration: 2 hours


I met up with the team in lab to discuss the power considerations of the project with the professor as well as to discuss some potential changes to our PSDRs. Alan and Juho had already decided on a new motor and LCD after the power and managed to get prototypes which match the interface so we can start working with the microcontroller next week. For the PSDR changes, the main consideration was removing the keypad PSDR and replacing it with a MIDI file pre-processing and conversion into usable format one. The main reason for this is that the MIDI file functionality has already been defined as a core part of our project and it will most probably be one of the harder sections so not having it as one of our core PSDRs seemed weird.


In the meeting with the professor we mainly discussed the new parts we planned to include, safety precautions for testing and checking if our PSDR change was valid. He gave us some current limiting power sources to use for testing so that we don't fry our test parts, he reinforced the need for feedback on our motors and he OKed the PSDR change. After the meeting with the professor, I reworked the website a bit to include the changed PSDR and to include a lot more information about our additional features. The main changes were keypato explain all of our features not covered by our PSDRs, to describe all our potential features that we'll implement if we get time and to phrase the new PSDR with sufficient detail.

Entry 2: -----------------------------------------------------------------

Date: February 12th
Start Time: 2pm
Duration: 3.5 hours

The main change this week was that our microcontroller finally arrived. To start off with, before the lab, I got the microcontroller and started working on setup and interfacing. We still don't have a motor or LCD yet so there wasnn't much to do with interfacing so I started on setup. Most STM tools are windows only so I started setting it all up on the lab computer. The lab computers come with system workbench preinstalled, but the current STM recommended IDE is STM32CUBE so I tried to get it installed. I also researched a bit more on the datasheet and reference manual for the microcontroller now that we had it and updated the course website to include these documents and to clean up some of the PSDRs. https://wiki.st.com/stm32mcu/wiki/STM32StepByStep:Step1_Tools_installation#Install_STM32CubeIDE - Installation link for STM32IDE
https://www.st.com/resource/en/user_manual/um1727-getting-started-with-stm32-nucleo-board-software-development-tools-stmicroelectronics.pdf - User manual for board Once the rest of the team arrived and the lab started, we unpacked the microcontroller and tried to get it on a breadboard. The NUCLEO kit is very weirdly designed and has some extra ground pins on the 4 corners. Each corner has 2 ground pins and they are all oriented at different angles which makes plugging it into the breadboards we had impossible. The spacing between the actual pins is also kind of inconsistent, with half the pins on the same column (on the same side of the bridge on the breadboard) having a small extra gap between them. We collected all the different breadboard types in lab and tried each of them individually as well as mixing and matching 2-3 of them to get this board in, but it wouldn't work. In the end, we decided to just use the flywires instead.

After that, we discussed our PSDRs again. The main issue was that we are only allowed 2 stretch PSDRs, but currently we had 3, adding rotary actuators to strike either the center or edge of the drum for different sounds, having a knob to change bpm and display it accordingly and having 4 drums instead of 2. After some discussion, we realised that MIDI files and general drumming sheet music doesn't specify where on the drum to hit so the rotary actuator PSDR isn't too necessary. We also really wanted 4 drums for a good final design presentation and bpm change is almost entirely software and not having it massively limits our range of songs so we wanted that as well. We also discussed all our PSDRs and our functional descriptions with the TAs and the professor and worked on the feedback. They OKed the PSDRs and found some information missing from our website such as stretch PSDRs missing from our additional features section so we looked into it. For the rest of the lab we worked on the bill of materials and discussing some issues the TAs found with our circuitry and plan. THe TA in charge of ordering parts told us that using a motor without feedback could cause the drumstick to slowly deviate from our given rotational values which could lead to the drumstick eventually missing the drum or trying to rotate further after it already hit the drum which could lead to the drum being damaged. The planned motors will also have a high current draw so having 4 of them working at once may not be feasible. With this in mind we decided to try out the motor the TA gave us last week even though we dismissed it last time due to the slow rotational velocity, to order one of the planned motor to try it out and lastly to try and find another low current alternative with feedback.

Entry 1: -----------------------------------------------------------------

Date: February 8th
Start Time: 1pm
Duration: 1 hour

Had a team meeting to discuss the electrical overview of the design, the system flowchart and the component analysis. Alan had finished the electrical overview so we all discussed it to clean up the document and make sure that we're all on the same page. The general system block diagram had the microcontroller interfacing with the LCD through SPI, the servos through PWM and the keypad through GPIO pins as well as the wall outlet being passed through a AC/DC converter to get the DC power needed by the motor, the LCD and the microcontroller and the resulting AC power being passed through a buck converter to get down to 8.4V for the servo and finally another buck converter to get it down to 3.3V for the LCD and the microcontroller. After we all understood the block diagram, we had a discussion and did some research on the exact operating voltage and current draw of some of the parts including the motor and the LED screen. Once that was dealt with we moved on to the component analysis. For the component analysis, Alan and Juho had already been working on tables for comparisons of different options for the motors, LCD and microcontroller so I just wrote a quick writeup to summarize all the findings. The excel docs were pretty well formatted and had the chosen parts in green so I just added pictures of the tables and then wrote the required specs and what we chose. We ended up with the STM32H723ZGT for our microcontroller, the BLS3355 for the motors, the EVE3-50A-BLM-TPN-F32 for the LCD and the FT234XD for the USB to UART IC.