NRF TXT on MeshCore

Posted on by

I got pulled into a new hobby: mesh radio. It’s using the 915MHz ISM band with LoRa. Pretty simple in concept. Small radios act as modems. Apps on phones use BLE to talk to the modem. Messages get repeated by other radios.

There are two main, competing projects. Meshtastic and MeshCore. I think Meshtastic is superior in almost all aspects except for one. It falls over with high density. As a result of that, it lost traction in urban areas like Seattle. So, annoyingly, MeshCore became the de-facto standard around me even though it’s worse in nearly every way.

I started out on this adventure when I bought this thing: https://www.etsy.com/listing/1873027341/nrf-txt-most-advanced-stand-alone

NRT-TXT

It’s this cute little device that was built for Meshtastic. It’s based on the Heltec T114 radio which is supported by MeshCore, but MeshCore is VERY phone-centric. In order to get this little guy to be a mostly standalone radio, I had to build a custom firmware.

After many hours of coding, vibe coding, drawing little icons, and testing. It’s finally working well enough to be usable-ish.

If you got one of these and want to try it on MeshCore, check out my FW project here. https://github.com/kallanreed/MeshCore/tree/nrf-txt-fw

You first need to add contacts and channels with the phone app but then you can send and receive messages using only the device. There are a few hidden UI things that are not intuitive so here’s a cheatsheet.

Global

  • FN+H: Go to Home screen.
  • FN+I: Toggle screen invert.
  • Any key: Wake the screen if it is off (the first keypress only wakes the display).

Home Screen Navigation

  • LEFT: Previous page.
  • RIGHT: Next page.

Lists And Prompts

  • UP: Move selection up.
  • DOWN: Move selection down.
  • ENTER: Select/confirm.
  • ESC: Cancel/close prompt.

Home Page

  • FN+R: Mark all messages read.
  • ENTER: Open Options prompt (Toggle Buzzer, Toggle Invert).

Recent Adverts Page

  • UP / DOWN: Select advert.
  • ENTER: Add contact (opens Add/Cancel prompt).

Contacts Page

  • UP / DOWN: Select contact.
  • ENTER: Open contact thread.

Channels Page

  • UP / DOWN: Select channel.
  • ENTER: Open channel thread.

Thread Screen (Contact Or Channel)

  • ESC or LEFT: Return to Home.
  • FN+ENTER: Compose message (opens text input).
  • FN+R: Mark thread messages read.
  • ENTER: Open selected message in viewer.
  • UP / DOWN: Move through message list.

Message Viewer

  • ESC or LEFT or ENTER: Return to previous screen.
  • UP: Previous message.
  • DOWN: Next message.

Text Input

  • ESC: Cancel and return.
  • LEFT / RIGHT: Move cursor.
  • ENTER: Send/accept input.
  • BACKSPACE or DEL: Delete character before cursor.

Radio Page

  • ENTER: Open Radio Options prompt (Reset Stats, Toggle BLE, 0-Hop Advert, Flood Advert).

GPS Page

  • ENTER: Open GPS Sensor prompt (Enable, Disable).

Power Page

  • ENTER: Open Shutdown prompt (Yes, No).

The OP-XY is Half-Baked

Posted on by

It’s pretty… but skin-deep.

I picked an OP-XY up for my birthday and it’s a lot of fun to play with but it’s certainly not worth the price. There are far too many missing features to justify the price. I’m going to list the feature gaps I run into while making songs below and hope that it either helps others decide whether it’s worth buying or maybe someone from TE will stumble across this and make some fixes.

I’m on the latest firmware 1.0.45

  • Recording to GarageBand over USB out is distorted.
  • No stereo sampling over USB input.
  • No per-note nudging when there is more than 1 note on a sequence step. (OP-Z can do this.)
  • No side chaining/ducking.
  • You can live record things that can’t be sequenced.
  • No chopping (on a sampler?!?!)
  • No timescale or tempo matching (the EP-1320 has this!! For $2000 less!!)
  • No fine control over note lengths. (All those knobs too…)
  • One LFO that is limited on what it can modulate. No mod-rate modulation either. Can’t drive the LFO from an Envelope.
    • The modulation matrix all up is very weak.
  • Parameter sequencing is very difficult to edit and results in lots of unhappy accidents when making changes.
  • Maestro has bugs where it will leave notes hanging.
  • Can’t start playback from the current bar. Very annoying when working on the 4th bar of a pattern.
  • No probability trigs. There’s random, but that’s not good enough for cases where you want real control over the pattern.
  • Similarly, there’s no “trig with previous” which would allow for small phrases to randomly trig.
  • Wavetable engine doesn’t support custom tables and there are only like 8 out of the box.
  • Bugs when setting note length. Especially when setting a second note on the same step (think sequencing chords.)
  • Sample management is a mess. MTP is a giant pain in the ass. You’re only able to have one layer of folders.
  • When opening a sample, it frequently gets confused by any numbers in the filename which causes it to set Tuning values that are completely incorrect.
    • Also there’s no quick way to reset the tuning to 0 so you’ll have to manually adjust with the knob.

Can you use it to make music? Yes. Could you make the same or better music for much cheaper. Double yes. The Deluge with the community firmware is $1000 cheaper and many times more capable as a sampler, synth, and sequencer.

UPDATE: I got tired of the bugs. Every new FW version came with new regressions. Too bad. I loved the device but the dev team ought to be ashamed (if not fired).

Building R packages from source on MacOS

Posted on by

For some reason Apple decided to break building C/C++ on MacOS by moving the C and C++ stdlib headers into a rando location that most toolchains don’t understand. Amazing. I ran into this while trying to build the R ggplot2 package from source and got missing header errors.

When installing a package from source fails, you’ll see a message like this.
Warning in install.packages :
installation of package ‘Rcpp’ had non-zero exit status

If you look back in the build output, you’ll see errors about missing headers. Something like this:
In file included from api.cpp:26:
In file included from ../inst/include/Rcpp.h:27:
In file included from ../inst/include/RcppCommon.h:30:
In file included from ../inst/include/Rcpp/r/headers.h:66:
../inst/include/Rcpp/platform/compiler.h:100:10: fatal error: 'cmath' file not found

The problem is that Apple moved these files out of /usr/include and /usr/local/include into a magical SDK package that most things don’t know about. Luckily R provides a way to set up the build environment when building packages from source with a file called Makevars.

First you need to find the location of your headers. Assuming you already have the Xcode command line tools installed, run xcrun --show-sdk-path
> xcrun --show-sdk-path
> /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

In that package, you’ll find usr/include — but that’s not where the C++ headers are either. They are actually in usr/include/c++/v1.

Next, you need to create the Makevars file ~/.R/Makevars — you may need to create the .R folder first. (mkdir ~/.R)

Once you’ve created the Makevars file, add this build variable and save:
CPPFLAGS = -I /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1 -I /opt/homebrew/include

Open an R command prompt and run tools::makevars_user() — you should see the path to your new Makevars file if you put it in the right place.
[1] "/Users/username/.R/Makevars"

You should be good to go.

UPDATE – I still ran into more problems while trying to install the devtools package and needed additional tweaks. For reference, I’m trying to follow the setup steps for the book “The Art of Machine Learning” by Maloff where the ultimate goal of all this mess is to get install_github("https://github.com/matloff/qeML") to install correctly.

  • brew install libgit2

My Makevars file now looks like this hack-a-palooza. The R source packages are pretty dirty and their build scripts seem to need a lot of help selecting C++ version especially.
INCLUDE_PATH = -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1 -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/opt/homebrew/include
CPPFLAGS = $(INCLUDE_PATH)
CFLAGS = $(INCLUDE_PATH)
CC = clang $(INCLUDE_PATH)
CXX = clang++ -std=c++17 $(INCLUDE_PATH)
CXX11 = clang++ -std=c++11 $(INCLUDE_PATH)
CXX14 = clang++ -std=c++14 $(INCLUDE_PATH)
CXX17 = clang++ -std=c++17 $(INCLUDE_PATH)
OBJCXX = clang++ -std=c++17 $(INCLUDE_PATH)

References:

Built-in Columns in JSON View Forms

Posted on by

SharePoint Online supports custom list view formatting with a funky little JSON DSL that is meant to allow customization without giving users full control over HTML or Javascript which is unsafe. This feature was added back in 2017.

The way it works, only displayed columns are allowed to be referenced in the custom JSON. Unfortunately, the feature owners never got around with supporting the “hidden” or “default” SharePoint columns like Created By or Last Modified. As a result, there is no way to show these columns at all in the “modern UI”. (As of 2024, this is still the case and it boggles the mind).

At work we have these “hack day”-style events that we call FHL (Fix. Hack. Learn.) where we are given some time to work on knocking out some of the little problems that get in the way of work, annoy us, QOL improvements, etc. This bug was bothering me because we are using some SharePoint lists internally to track stuff and we often want to see who created an item in the list, but there was no way to show it without some crazy hacks. (Dear feature owners: using a workflow to copy a default field into a totally redundant one on the same item is not my idea of a workaround.)

Long story short, I added hacky support for some of the default columns so they can be used in custom JSON views. The following columns should be able to be referenced now in SharePoint Online.

  • $Author — The “Created by” user. This a user object, e.g. [$Author.title] or [$Author.email]
  • $Editor — The “Modified by” user.
  • $Created — The date the item was created.
  • $Modified — The date the item was last modified.
  • $ID — The list item id.
  • $_UIVersionString — The “pretty” string for the current item version.

Here’s the most basic demonstration of adding a custom footer to the list view form for a document.

{
    "elmType": "div",
    "style": {
        "display": "block"
    },
    "children": [
        {
            "elmType": "div",
            "txtContent": "[$Author.title]"
        },
        {
            "elmType": "div",
            "txtContent": "[$Created]"
        },
        {
            "elmType": "div",
            "txtContent": "[$Editor.title]"
        },
        {
            "elmType": "div",
            "txtContent": "[$Modified]"
        },
        {
            "elmType": "div",
            "txtContent": "[$ID]"
        },
        {
            "elmType": "div",
            "txtContent": "[$_UIVersionString]"
        }
    ]
}

Here are a few posts from others online who have been struggling with this. Hopefully they run across this post and get unblocked.

ETW and Async in .NET

Posted on by

I ran into this problem at work and wanted to capture the findings for future me.

ETW uses the ActivityId field on the TEB to track an activity. This gets logged with the ETW events.

When adding async Tasks to the mix, Tasks have an abstraction on the execution context that is per Task. Tasks may run on multiple different threads and so it’s possible that a Task gets hooked up to a thread that already has an activityId set. This messes up logging.

AsyncLocal provides a way to “flow” an ambient value across multiple Tasks as part of a logical operation. This can be used to track activityId on Tasks, but the trick is getting it pushed down onto the native thread.

AsyncLocal has a constructor argument that is a callback that is executed when the value changes. Values are always refreshed from empty when a Task is assigned to a thread. This provides a place where the activityId can be flowed down to the executing thread so that ETW logging works.

See ActivityTracker.cs on ReferenceSource to see how this works in .NET Framework.

Sonicare UART

Posted on by

The seal on my toothbrush failed and the driver isn’t working well anymore. I took it apart to see if I could fix it, but the driver is all corroded and is a model that’s not serviceable. Boo.

I figured I’d scavenge the battery. While I was looking at it, I noticed that there are UART and I2C pads. Interesting. I soldered on some wires on and tried the Flipper Zero UART console. Sadly, nothing lit up.

I wonder what a toothbrush has to say…
Maybe the dorkiest thing I did over break?

I hooked up my little oscilloscope and checked all the pins. The UART pins seem to be pulled high and the I2C pins seemed to float. There was no clock or anything that looked like signal. Strangely, the UART pin voltage was 4.7v.

I could mess around with it some more, but I think it’s probably just there for programming the chip though the board at the factory.

Dirty-PocketChip-wave M8

Posted on by

Before I decided to buy the M8, I tried it “headless” with a Teensy for a while to see if I could figure it out. It worked well enough, but the whole point of the M8 is that it’s portable and fun. I thought I’d try to get it running on my PocketChip. It worked and it wasn’t even that bad to use. If I really wanted to I could have stuffed the Teensy into the Chip’s case and had a pretty decent portable experience. I even made a song on it.

I wrote a short readme and have the source on my GitHub. There’s also a pre-built binary if you have a Teensy and a PocketChip and want to give it a try.

Setting a Person field in SharePoint

Posted on by

This shouldn’t be this hard. The SharePoint team really needs to work on the programming model. This web API is bad bad.

For “reasons” I found myself needing to update a SharePoint list item to set the value of a Person field. It turns out that this is insanely annoying because what the ListItem API expects is the resolved User Id (it’s some rando number that’s assigned to the user when they access the site for the first time.) I’m working in a PowerAutomate flow and unfortunately that’s not a value that comes along with a user’s object. That would be too helpful.

If you try to use the Claims string to set the PersonFieldStringId field in a MERGE, it just doesn’t work. You get some terrible error message like Bad Gateway or something equally useless. Also, if you try to set through the navigation property like PersonField: { "Claims": ... }, that will succeed, but not actually work.

After being very frustrated and increasingly annoyed, I finally found someone who said to use the validateUpdateListItem method on the item. It worked. Apparently, this method “Validates and sets the values of the specified collection of fields for the list item.”

In case you find yourself on this site, wondering how this can all be so unnecessarily complicated, here’s the request you want to make.

URI: https://tenant.sharepoint.com/sites/MySite/_api/web/lists/GetByTitle('My List Name')/items(1234)/validateUpdateListItem
Method: POST
Headers: Content-Type=application/json;odata=verbose
Body:
{
  "formValues": [
    {
      "FieldName": "Your_x0020_Field_x0020_Name",
      "FieldValue": "[{'Key':'i:0#.f|membership|user@domain.com'}]"
    }
  ]
}

Radio Adventures

Posted on by

After my year of music I got a Flipper Zero kind of on a whim. I didn’t know much about it but it was kind of fun trying to catch signals. The biggest problem was that it was limited to digital signals inherently and the SubGHz bands it supports are kind of uninteresting unless you’re into hacking. I liked the portability and the slick case, but generally it doesn’t really fit into any use case. I wrote a little IR scope that lets you visualize an IR signal. That was kind of fun.

Working on the PortaPack firmware

Then I was introduced to SDR and more precisely the PortaPack built on HackRF. That’s the flexibility of a fairly powerful, half-duplex SDR, in a portable device. Even better is that it has a great open source community. The Mayhem project was a lot of fun to work on. Lots of interesting problems to solve, bugs to fix. The community has some great people too. It’s all C++ so it was nice to have a place to continue to practice my coding.

I spent hours and hours and days and weeks working on the project, but then, as all my interests do, I found something else to focus on. In this case it was the Dirtywave M8. Does that mean it’s back to making music? Maybe. That and a renewed focus on home improvement projects…

Long time; No post

Posted on by

I just noticed my last post was in 2021. What have I been doing lately? In the spring of 2022 I got into making music. I started out with a Teenage Engineering P.O. 20 and made a few songs. Liked that so I got a P.O. 133 which is a little sampler. That was fun, but the sound quality was meh. Then I bought a Novation Circuit Rhythm. That thing was awesome, but working with samples was kind of limiting so I wanted a synthesizer. So I got a Modal Skulpt which was fun. Then I wanted more power so I got an Arturia MicroFreak. I wanted to like that more than I did, so I sold that. I got a Novation Circuit Tracks which was great, but working with the synths was a pain in the butt because it was all done on the computer. Then I wanted even more synth power so I got an ASM Hydrasynth Explorer. That thing is epic. Love it. The Novations weren’t cutting it so I sold them both and got an MPC One, which is super powerful, but it has some warts (that piano roll is painful.) Finally, I ended up with a Dirtywave M8 which I strangely love. It’s so tedious to use, but I love how small and portable it is and the synth engine is actually pretty nice sounding.

Dirtywave M8

Along the way I dabbled in various software. Sunrizer Synth is really great. Koala Sampler is also fantastic and is a great replacement for the P.O. 133 at a fraction of the cost. It’s 100% worth paying for the full version. I also dabbled with Synthesizer V and AI singing. Super cool and I need to open that app up again (it was pretty pricey.)

Check out my music on SoundCloud.

2023 was the year of radio. It started with a Flipper Zero and ended up being a maintainer on the PortaPack Mayhem project. More on that in another post.