Steven Harman

Configuring SSH Keys for Multiple GitHub Accounts

Wed, 21 Aug 2024 00:00:00 +0000

Managing different SSH Keys for different Hosts is well-understood. Different keys for the same host (e.g., github.com), based on which GitHub Organization we’re working with, that’s… a bit trickier. But, we have the technology!

We’re going to configure our SSH and Git clients to seamlessly switch SSH keys between our personal GitHub Account and our GitHub Enterprise Cloud, EMU Account.

You need the right key.

🔐 Typical Multi-Key Setup

Most guides and information we’ll find on the Internet about managing multiple SSH keys focus on multiple Host values. As in, use key A for example.com and key B for stevenharman.net. A simplified ~/.ssh/config for that might look something like:

Host example.com
  IdentityFile ~/.ssh/id_a_ed25519

Host stevenharman.net
  IdentityFile ~/.ssh/id_b_ed25519

🤹 Different `HostName`s, Different Keys

But what happens when the HostName, i.e., the real host name that will be used for the SSH connection, for both is the same? For example, when we have a personal GitHub Account (for Open Source work, some private repositories, and maybe even some repos belonging to other GitHub Organizations), and second GitHub Enterprise Cloud Account provisioned by our employer.

For brevity, let’s say our personal GitHub Account is fry. And our work for Planet Express, Inc., has a GitHub Enterprise Cloud instance, and provisioned us a fry_plnx Enterprise Managed Users Account.

In both cases, the SSH HostName is github.com. We might try a configuration like this:

Host github.com
  HostName github.com
  User git
  IdentityFile ~/.ssh/id_fry_ed25519

Host github.com
  HostName github.com
  User git
  IdentityFile ~/.ssh/id_fry_plnx_ed25519

This configuration will work for git talking to GitHub for the fry Account, because it is matched first. But when working with a repository for the fry_plnx Account, the first Host github.com entry matches, and uses the id_fry_ed25519 SSH Key. However, that key is not associated with the fry_plnx Account on GitHub and so… we see errors.

$ git clone git@github.com:planet-express/delivery_service.git
  ERROR: Repository not found.
  fatal: Could not read from remote repository.
  
  Please make sure you have the correct access rights
  and the repository exists.

🤡 Use Different `Host` values

At this point, most articles on the Internet will tell us to override the SSH command used to connect, or (IMO, slightly better) use different Host values for each. The later would look something like this:

Host github.com
  HostName github.com
  User git
  IdentityFile ~/.ssh/id_fry_ed25519

Host github-plnx
  HostName github.com
  User git
  IdentityFile ~/.ssh/id_fry_plnx_ed25519

A big downside to this technique is that any time we’re cloning a new repository for our fry_plnx Account, we’ll need to adjust the URL used. Meaning we can no longer copy/pasta the command from the GitHub UI. Nor rely on muscle memory we’ve built up over years of using our personal Account.

# Instead of the actual URL
$ git clone git@github.com:planet-express/delivery_service.git

# Substitue in our custom Host value for the `github.com` part
$ git clone git@github-plnx:planet-express/delivery_service.git

And this will work! But we have to remember to manually substitute those values for each repository we clone. It’s toil. And I detest toil.

🤖 Automate Substituting the `Host`

I happened across an article using Git’s url.<base>.insteadOf variable to swap URLs. We can use that to teach Git about our Host github-plnx entry, and have it auto-magically swap github.com back in when talking over SSH to GitHub. We’ll add this to our ~/.gitconfig:

# See custom `Host github-plnx` in ~/.ssh/config
[url "github-plnx:planet-express"]
  insteadOf = git@github.com:planet-express

By including the planet-express GitHub Organization name in the match for the url variable, Git will alter the URL git@github.com:planet-express/delivery_service.git by matching github.com:planet-express and swapping in github-plnx:planet-express. Then the SSH config matches the host github-plnx, and using that matched config, replaces github-plnx with the actual user (git) and host name (github.com). At which point we’re back to the original URL. 🎉 NEAT!

With that in place, Git commands for repositories under github.com/planet-express will use the id_fry_plnx_ed25519 SSH key. Any other repositories will fail to match the github-plnx host, thus using the regular Host github.com SSH Config, and the id_fry_ed25519 key.

NOTE: We’d need to add a similar entry for other Organizations in the GitHub Enterprise Cloud instance that we need to work with.

🧹 Keeping `.gitconfig` Clean

If you’ve got your dotfiles in Git, you might want to keep employer and/or machine-specific configuration out of the ~/.gitconfig file. For example, our work machine needs to know about the custom url "github-plnx:planet-express" Git config, but our personal MacBook likely doesn’t! Or maybe we don’t want to, or cannot, expose every GitHub Enterprise Cloud Organizations we’re a part of in our dotfiles repo.

Luckily, we can leverage Git’s ability to include other config files to keep such customizations out of Git. We can adjust the ~/.gitconfig file, which is in our dotfiles Git repo, to include another config file which is NOT in Git. Add the following to the bottom of the ~/.gitconfig file:

[include]
  path = ~/.gitconfig_custom

If the ~/.gitconfig_custom file isn’t present, Git just ignores the include directive. As is the case on my personal MacBook. 😅 Phew!

When it is present, Git will load the file, allowing us to override or add additional configuration. Now, move the [url …] configuration from ~/.gitconfig into the new ~/.gitconfig_custom file:

[url "github-plnx:planet-express"]
  insteadOf = git@github.com:planet-express

And voila! We are switching SSH keys automatically, and our ~/.gitconfig is clean of any knowledge about the specific GitHub Organizations, etc…

On Flaky Tests, Time Precision, and Order Dependence

Mon, 20 May 2024 00:00:00 +0000

A flaky test is one that fails unpredictably, without a corresponding change in the code under test. These often show up in CI runs where a test unrelated to any change made suddenly fails, and then mysteriously passes when re-run. There are many type and causes of “flakes”. Today I want to talk about flaky tests caused by time precision, and time-order dependence.

The specific tools mentioned here will be Ruby, Rails, and PostgreSQL specific. But analogs exist in nearly all other languages and frameworks, so the techniques are broadly applicable.

Not sure if I broke something, or it's a flaky test…

🥶 Freeze Time

Sometimes a test assumes that time will elapse (or not elapse) during the execution of the test. In such tests, when comparing time/timestamps, try to work with entirely predictable time. Ideally by making time… umm… more static.

In RSpec, you can ensure that Time.now and friends always return the same value during a single test execution by freezing time with Rails’ freeze_time helper. From the docs:

Time.current #=> Sun, 09 Jul 2017 15:34:49 EST -05:00
freeze_time do
  sleep(1)
  Time.current #=> Sun, 09 Jul 2017 15:34:49 EST -05:00
end
Time.current #=> Sun, 09 Jul 2017 15:34:50 EST -05:00

I prefer to add a bit of RSpec metadata to do the magic. 🪄

# spec/spec_helper.rb

config.around(:each, :time_frozen) do |example|
  freeze_time do
    example.call
  end
end

With that, you can add the :time_frozen metadata to a wrapping context or describe block:

RSpec.describe MyClass, :time_frozen do
  # Time is frozen here… 🥶
end

Now all references to Time.now and friends are frozen. Time will not move forward inside that block.

It’s important to know that any setup (like RSpec let!, before, or around blocks) happening before the time-frozen block did not have time frozen, and so can have slightly different values Time.now than inside the block. We’ll come back to this later. 😉

⏱️ Platform-specific Time Precision

Ruby’s Time and DateTime objects have up to nanosecond precision. That is, up to 9 significant digits after the decimal point. e.g. Time.utc(2024, 5, 20, 0, 0, 0.123456789) would result in a Time object with 123,456,789 nanoseconds of precision.

However, the precision of Time.now (and friends) varies by platform. On macOS, it’s limited to microseconds - 6 significant digits after the decimal point. Linux tends to be nanoseconds.

This usually doesn’t matter because any single test run happens on the same machine. However, there’s a wrinkle: PostgreSQL. Or, more generally, the precision of time-like data in your database.

PostgreSQL’s time, timestamp and interval data types have a resolution of microseconds. And MySQL has a similar limitation on precision. So during a CI run, for example, it’s possible to do something like this:

now = Time.now #=> 2024-05-20 19:58:45.660142789 +0000
BlogPost.create(published_at: now)

post = BlogPost.last
expect(post.published_at).to eq(now) #=> Flakes b/c post.published_at lost resolution

post.published_at #=> 2024-05-20 19:58:45.660142000 +0000

In my experience, the need for directly comparing to a time field from the database is rare. But in those rare cases, I prefer to #round the in-memory time to the same precision as the database column.

expect(post.published_at).to eq(now.round(6))

Another option is more fuzzy expectations/matching. RSpec has the be_within matcher, for example. Though I find this more confusing than helpful, and tend to avoid it.

🤹 Time-order Dependent Tests

In that contrived example it’s easy to see how we end up with mismatched precision. A more tricky variation occurs when we create a bunch of test data and then make assertions on that data, based on the order of various time fields.

posts = FactoryBot.create_list(:post, :published) #=> Generate Post's with #published_at = Time.now at time of creation
sorted_posts = posts.sort_by(:published_at).reverse #=> sort the in-memory list, "newest" first
recent_posts = RecentPostQuery.call
expect(recent_posts).to eq(sorted_posts) #=> Flakes

In this case, the posts collection was held in memory, with each Post#published_at value having full nanosecond precision. When saving to PostgreSQL, the precision was reduced to microsecond. Meaning it’s possible two Posts were created fast enough that when sub-microsecond precision is lost, they have the “same” #published_at in the database. Sorting equal values is non-deterministic, so Post instances with the same #published_at can have different ordering within the recent_posts collection. And so… flaky test failures.

A naive way to handle this would be to #reload the posts collection before sorting it in-memory. Though that can result in the same problem of loss of precision, with the in-memory collection having non-deterministic ordering.

A robust way to solve this is to use explicit Time/DateTime values when creating the test data. After all, the ordering-by-time is actually the thing we’re trying to test. Being more explicit about that in test setup helps our future selves understand which parts of setup are important, vs incidental.

oldest = FactoryBot.create(:post, published_at: 2.days.ago)
middle = FactoryBot.create(:post, published_at: 1.day.ago)
newest = FactoryBot.create(:post, published_at: Time.now)
recent_posts = RecentPostQuery.call
expect(recent_posts).to eq([newest, middle, oldest]) #=> works every time!

🥶⏱️ Precision of Frozen Time

Using Rails’ built in #freeze_time, #travel_to, etc… will result in all sub-second precision being lost, by default. This can be surprising, or at least unexpected. I know it surprised me!

As with freezing time in general, this loss of precision usually doesn’t matter. However, it can cause issues similar to those above where a loss of precision leads to non-deterministic ordering. Consider this terribly made up example:

let!(:post) { FactoryBot.create(:post, :published) }

context "with a draft post" do
  let(:draft) { FactoryBot.create(:post, :draft) }

  before do
    freeze_time
  end

  it "includes a published draft as the newest Post" do
    draft.publish! #=> sets published_at = Time.now
    recent_posts = RecentPostsQuery.call

    expect(recent_posts).to eq([draft, post]) #=> Flakes
  end
end

This can be tricky to spot.

Here the let!(:post) happened before we froze time, so it might have a #published_at = 2024-05-20 0:0:0.123456000 +0000. Then, a fraction of a second later (because computers are fast), we freeze time, create the draft, and #publish! it. Since we know the creation and publication of the draft happens after the creation of post, this setup seems to make sense.

Except frozen time has no sub-second precision. So it’s possible that thedraft’s published_at = 2024-05-20 0:0:0.000000 +0000. Which is a fraction of a second before the post’s, rather than after it.

We might try reaching for our :time_frozen RSpec helper here, relying on RSpec’s behavior to ensure all before blocks (which include let!) run before the actual test code (it block). And indeed that is the case. But this is also subtly broken as now all of the times lack sub-second precision. Meaning we’re back in the non-deterministic ordering problem.

let!(:post) { FactoryBot.create(:post, :published) }

context "with a draft post", :time_frozen do
  let(:draft) { FactoryBot.create(:post, :draft) }

  it "includes a published draft as the newest Post" do
    draft.publish! #=> sets published_at = Time.now
    recent_posts = RecentPostsQuery.call

    expect(recent_posts).to eq([draft, post]) #=> Flakes b/c time's can be equal
  end
end

The solution, like before, is to be explicit about time values when ordering based on them is important to the test.

Preserving sub-second precision

With newer versions of Rails, sub-second precision can optionally be preserved using the with_usec option. You need to opt into this at each freeze_time, travel, and travel_to call site, but at least it’s possible!

freeze_time(with_usec: true) do
  # Time here is frozen, and includes sub-second precision.
end

🎁 Wrapping up

In summary, take care when directly comparing or sorting on time (or time-like) values. And when setting up test data where time is important, be explicit about the exact time values being used.

Good luck. And happy testing!

🙏 Thanks

Thank you to Hazel for feedback and proofing!

So We've Got a Memory Leak…

Thu, 25 Jan 2024 00:00:00 +0000

Memory leaks happen. And if you’re here, reading this, I’d bet you’re dealing with one. First things first - are you sure it’s a leak, and not bloat?

Okay, so it’s a leak. Much has been written about various tools for profiling a leak, understanding heap dumps, common causes of leaks, work being done to improve Ruby’s memory layout, and so much more. Ben Sheldon’s recent “The answer is in your heap: debugging a big memory increase in Ruby on Rails” is a great example. Ben mentions and shows how he and some teammates used several different tools to generate heap dumps, analyze and interrogate them, and ultimately find and fix the source of a leak in Rails itself.

These are all great resources, and can be crucial knowledge in our hunt for a leak. But they all suppose we already have an idea of where to start looking, or use stripped down examples to showcase the tools. The memory_profiler Gem can profile memory allocations around some suspect code, if we already know that code is suspect. Derailed Benchmarks can get us started on tracking down a leak in the overall stack, or a known problematic resource, and generate heap dumps for us. Comparing those dumps with heapy can point us in the right direction by revealing memory being unexpectedly retained over time. We can use sheap to track down exactly where problematic objects allocations are happening, once we’ve identified those problematic objects.

But what if we’ve reviewed all recent code changes, and nothing stands out? Or if the leaks don’t happen consistently, across all instances of the running app? Or memory starts leaking different times? Where do we even start?

That’s what I was asking myself recently with a decade old Rails app I work on. This is a real production app, driving substantial revenue. It serves a sustained load of 400-500 requests/second, with peaks into several thousands/second.

Then one, nondescript day, amongst the usual flow of deploys, memory started spiking. 📈 The pager started yelling. 📟 And we had what looked like a memory leak. 💧

We have a memory leak! So. Many. Spikeys!

🔥 First put out the fire

It certainly looked like a leak, rather than bloat, meaning we had one sure-fire way to temporarily fix the problem. Restart things! We run on Heroku, and what we’re seeing above are per-Dyno memory numbers. In the short term, we’d either let our normal cadence of many-deploys-a-day restart our web instances, or we’d manually restart individual Dynos approaching their memory limit.

🔎 Look for suspect changes

With the fire under control we turned our attention to changes that happened in the hours before the first big spike. The graph above doesn’t show our deploy markers (they were too distracting at this granularity) but starting with the deploy just inside of the first spike and working backward, we audited all code changes going back three days.

We found a few changes that seemed like they could be related. One of them introduced a memory leak in development mode due to Rails’ code reloading. Another caused us to hit Redis more than we intended during certain request filtering. The third looked like we were making more database calls, loading more ActiveRecord instances than we wanted (a classic N+1). We fixed the first two and rolled back the third, deploying them one at a time. The memory leak continued. 📈

No other code changes looked like they could cause a leak. But, we had made a few tooling changes to start collecting Ruby language and Puma pool usage metrics, so we backed those out and deployed. The memory leak continued. 📈

🧩 Look for patterns

Next we went looking for patterns to the memory growth. Of note, it was only our web Dynos that were leaking. All of our Sidekiq and Delayed::Job (yes, both, for… reasons lost to history) Dynos looked just fine. Also, not all web Dynos were always leaking memory. Sometimes they’d go hours with relatively flat-ish memory usage consistent with a long-running web process. Then one, or a few, or all of them would start to leak memory. This made us suspicious that perhaps the leak was specific to a certain kind of traffic, rather than a problem in the stack, or with volume of traffic.

Low-traffic period, just post deploy. Hours pass before 2 of 4 Dynos start to leak memory.

When we looked at our Puma workers, we also saw that not all workers on a single Dyno were leaking. We run Puma in clustered mode, with 12 worker processes for the 8 vCPU on each Dyno. It wasn’t unusual to see just a couple of the 12 processes were using up nearly all of the memory on the Dyno. i.e., a couple workers were leaking, while the rest were fine.

Due to aggressive sampling we were unable to use our OpenTelmetry Traces to match specific kinds of traffic to specific Dynos at times that correlated to the start of the leaks. But we still had a strong suspicion the leaks had something to do with particular traffic. One of the smart folks I work with had the idea to try correlating via our logs - which are not sampled. But the tooling there didn’t make that particularly easy. So we put that idea in our back pocket and moved on to another.

🚛 Go to the dumps!

If memory is being leaked, looking at dumps of Ruby’s memory heap is probably a good idea. After all, all of those very smart Internet Folks wrote lots of words about it.

Step 0: Get the tooling in place

We use rbtrace to attach to a running Ruby process, meaning it needs to be loaded into the running process. Basically, make sure it’s in the Gemfile. It’s generally fine to have this loaded in production. But we’ve made it somewhat toggle-able; we can opt in/out of loading rbtrace via an Environment Variable:

gem "rbtrace", require: String(ENV.fetch("FEATURE_ENABLE_MEMORY_DUMPS", false)) == "true"

Step 1: Find a leaking process

Running on Heroku meant we needed to dump the heap of a live, running web process, from within a Dyno itself. Luckily, Herkou and the Ruby community have already built all of the tooling necessary.

Using heroku ps:exec to open an SSH tunnel into a leaking Dyno, we needed to find a particular Puma worker process to dump. We went to our old *nix friend ps.

ps -eo pid,ppid,comm,rss,vsz --sort -rss | grep ruby

This gave us a list of all of the Ruby processes running on the Dyno, sorted by their RSS (amount of memory being used by the process). In the case of a web Dyno, most of them have the same PPID. These are the Puma workers. The process with a matching PID is the Puma primary (master) process. It will look something similar for Sidekiq. Other Ruby tools that run only a single process would look a little different. Picking the process with the most memory (i.e., the leakiest of them) we use its PID to and rbtrace to start tracing memory allocations.

Step 2: Enable memory allocation tracing

We set that worker process ID as $DUMP_PID to be used as a variable in later steps. This helps prevent silly typos and much face-into-keyboard-ing later on. 🤦

DUMP_PID=<pid>
# Turn on allocation tracking in the Ruby process.
# This will impact performance; it can use a lot of memory and CPU
rbtrace --pid="${DUMP_PID}" --eval="Thread.new{require 'objspace';ObjectSpace.trace_object_allocations_start}.join"

Step 3: Dump the heap

# Generate the first heap dump
rbtrace --pid="${DUMP_PID}" --eval="Thread.new{require 'objspace'; GC.start(); io=File.open('/tmp/heap-${DUMP_PID}.json', 'w'); ObjectSpace.dump_all(output: io); io.close}.join" --timeout=600

The .json file can be quite large. On a leaking process that had been running for a few hours, it was 5-6GiB in size. I’d recommend gzip-ing it.

gzip "/tmp/heap-${DUMP_PID}.json"

That will replace the .json dump file with a .json.gz file which could be up to an order of magnitude smaller.

Step 4: Fetch the dump

This will differ depending on where the application is deployed, but on Heroku we run the following (adjusting for exact file and Dyno names) from a shell on our local machine.

heroku ps:copy /tmp/heap-<pid>.json.gz --dyno=<web.1> --app=<app name>

Decompress and move the file somewhere organized. I’d recommend renaming it based on which dump this is; we’ll grab at least three if we’re going to use heapy to try to find retained memory, for example.

mv heap-<pid>.json.gz ~/dumps/.
gzip --keep --decompress ~/dumps/heap-<pid>.json.gz

Repeat steps 3 and 4 to grab as many dumps as needed.

Step 5: Clean up after ourselves

Be sure to turn off the allocation tracking, and probably remove those dumps from the Dyno. Or just restart the Dyno.

rbtrace --pid="${DUMP_PID}" --eval="Thread.new{GC.start;require 'objspace';ObjectSpace.trace_object_allocations_stop;ObjectSpace.trace_object_allocations_clear}.join"

🧐 Analyze the heap dumps

We tried looking at the retained memory reports via heapy, and meandering the diffs via sheap, but with no particular spot to start looking, it felt like looking for a specific needle in a stack of needles. (Thanks for that line, Tom Hanks.)

In hopes of seeing something that looked off, I went in search of a way to visualize the heap dumps. And The Internet™ did not disappoint! reap is a tool for analyzing a Ruby heap dump’s reference graph, and generating visualizations. Including flame graphs! Here’s a graphs of the third dump we took off of a leaking process, per the steps above.

reap path/to/heap-<pid>-2.json --count=15 --flamegraph=dump-2.svg

The way to read this is starting at the top and moving downward are references from a “root” (in the view of the Ruby Garbage Collector) to other objects. That is, objects further down are references by objects above them. And the more memory an object (or an object it references) is holding onto, the wider the cell in the graph.

Flame graph showing a Thread holding 1.9GiB of memory.

Notice anything odd? What’s up with that Thread holding 1.9GiB of memory? Or really, it’s the Array at the bottom, holding onto references to 32,067 other objects, in turn holding 1.9GiB of memory. So what the heck is that array? Where did it come from? Why is it referencing so much memory?

Flame graph calling out an Array of 32k things, referencing 1.9GiB of memory.

🧗 Go spelunking with `sheap`

Using the latest main branch from sheap (because some of the features like find_path weren’t yet released to RubyGems), we compared the second and third dumps. It will take a while to parse these dumps, which were pushing 6GiB in our case. ☕️

~/code/jhawthorn/sheap/bin/sheap heap-<pid>-1.json heap-<pid>-2.json

That opens an IRB console where we can explore the dumps and the diff between them. See the README (and the source code) for all that it can do.

irb: t = $after.at("0x55adecb68498")
===> <DATA 0x55adecb68498 VM/thread (137 refs)>
irb: $after.find_path(t)
===> [<ROOT vm (4014 refs)>, <DATA 0x55addb8deff8 ractor (31 refs)>, <DATA 0x55adecb68498 VM/thread (137 refs)>]

We first investigated that Thread to understand where it was coming from. Our suspicion was that it was a background thread from one of our telemetry or metrics tools - maybe the OTel tracer, or error reporter, or something like that? But, sheap told us it was a Puma thread - the one actually working requests, meaning it’s “the Rails thread.” So that question is answered.

Next we wanted to know more about that ActiveSupport::SubscriberQueueRegistry object. We went looking at the stable branch for the version of Rails we’re currently on (6.1) and found that it’s basically a per-Thread Hash used to store lists of ActiveSupport::Subscriber instances, based on the event name. And ActiveSupport::Notifications::Event objects are pushed onto and popped off that per-thread, per-event-name list (Array) as ActiveSupport::Notifications.instrument blocks are run. Going back to sheap we can see that in action - the SubscriberQueueRegistry instance holds a reference to a Hash, which has references to some String and Array objects.

irb: asr = $after.at("0x55adeeb31798")
===> <OBJECT 0x55adeeb31798 ActiveSupport::SubscriberQueueRegistry (1 refs)>
irb: asr.references
===> [<HASH 0x55adeeb31748 (6 refs)>]
irb: asr.references.first.references
===>
[<STRING 0x55adeb01afb0 "ActiveRecord::LogSubscriber-9300">,
 <ARRAY 0x55adeeb314f0>,
 <STRING 0x55adf09ffaf8 "Lograge::LogSubscribers::ActionController-276520">,
 <ARRAY 0x55adf09ffb20 (1 refs)>,
 <STRING 0x55adf4a029e8 "Lograge::LogSubscribers::GRPCWeb-276540">,
 <ARRAY 0x55adf4a02a10>]

The only Array with anything in it was in the middle of the flame graph with addresses ending ffb20. It held a reference to a single ActiveSupport::Notifications::Event instance. And that instance then held a reference to the Array of 32k+ other objects. If we look at the code for that Event class, we see it contains a single @children ivar Array. So, it’s an Array of what then? Child Event instances, presumably. We grabbed a couple to confirm.

irb: event = asr.references.first.references[3].references.first
===> <OBJECT 0x55adf4fdda70 ActiveSupport::Notifications::Event (4 refs)>
irb: children_ary = event.references.last
===> <ARRAY 0x55adf4fdda20 (32067 refs)>
irb: children_ary.references.first
===> <OBJECT 0x55adf523e9e0 ActiveSupport::Notifications::Event (4 refs)>
irb: children_ary.references.first.references
===> [<STRING 0x55addbe19ae0 "redirect_to.action_controller">, <HASH 0x55adf523e9b8 (10 refs)>, <STRING 0x55adedec2188 "19b24a9862f0e692859e">, <ARRAY 0x55adf523e990>]

Sure enough, the huge Array is a list of “children” Event objects. This first child Event was for a redirect_to.action_controller event name. And it would have been pushed onto this Event#children list because some other Event was already on the event_stack.

At this point our intuition was telling us something was wrong here - these Events were incorrectly being put on this #children Array. We wanted to know what else that child Event had to tell us, like what is in that Hash it’s holding a reference to?

irb: first_child = children_ary.references.first
===> <OBJECT 0x55adf523e9e0 ActiveSupport::Notifications::Event (4 refs)>
irb: first_child.references[1].references
===>
[<OBJECT 0x55adf4e54e38 ActionDispatch::Request (10 refs)>,
 <STRING 0x55ade7965bf0 "PresentationController">,
 <STRING 0x55adf4250150 "view">,
 <HASH 0x55adf4f6d4f0 (507 refs)>,
 <OBJECT 0x55adf4f5c998 ActionDispatch::Http::Headers (1 refs)>,
 <STRING 0x55adf4cccca0 "GET">,
 <STRING 0x55adf4f6e8a0 nil (1 refs)>,
 <HASH 0x55adf5378c70 (1 refs)>,
 <ARRAY 0x55adf5378ae0 (2 refs)>,
 <OBJECT 0x55adf5382310 (0x55adf5400058) (4 refs)>]

Whoa, an entire ActionDispatch::Request object. 😱 The request that was being redirected. What did that look like? Maybe we could reproduce this locally?

irb: first_child.references[1].references.first
===> <OBJECT 0x55adf4e54e38 ActionDispatch::Request (10 refs)>
irb: first_child.references[1].references.first.references
===>
[<HASH 0x55adf4ccced0 (222 refs)>,
 <HASH 0x55adf4f6d4f0 (507 refs)>,
 <STRING 0x55adf4f6e8f0 "/view/abc123def456?parked=junk&parking=junk&parseSchema=junk&… eliding more param tampering … &password=[FILTERED]& … &portbl=junk">,
 <STRING 0x55addc723e60 "https://">,
 <STRING 0x55adf4cccca0 "GET">,
 … # eliding more stuff
 <OBJECT 0x55adf4f5c998 ActionDispatch::Http::Headers (1 refs)>]

Well look at that. It‘s a mostly valid request, to an actual Route in the app, with a valid public ID of a resource (changed to protect the innocent, of course). What’s not legit is the parameter tampering - the work of a scanner very likely. What especially stood out was the password=[FILTERED] , which means something was catching the request and cleaning sensitive info we’d not want in logs and such.

🔄 Reproduce it

With the questionable looking query path and params, we opened the production app in an incognito browser window and made a request. We were met with a 500 server error.

🎉 Success! ✨

Logs confirmed that it was an error. Specifically a URI::InvalidURIError. The logs also recorded which Dyno the request had hit. We popped open our memory graphs and by sheer luck, that Dyno was currently showing normal memory usage. We put a temporary deploy freeze in place so we could monitor memory usage for a few minutes and before long the trend line was clear. We had a leak.

With some confidence we were leaking Event objects as part of the ActiveSupport::Notifications tooling, we still wanted to reproduce it locally so we could step through the code and see what was going on. Leaning on a little bit of binding.pry and even more puts debugging in the activesupport Gem (if you don’t know, bundle open <gem name>, check it out), we eventually reproduced the situation locally. And with a backtrace!

The error’s backtrace pointed to the uri Gem (part of Ruby’s standard library), which was being used by Bugsnag’s Bugsnag.cleaner.clean_url method, as part of a ActiveSupport::Notifications.subscribe block. Which sounded awfully familiar. A little more debugging and stepping through the Bugsnag and ActiveSupport code and We’d narrowed in on the problem(s).

Logs showing a Event object left on the stack after the error is raised.

💡Understand the cause

There were two related problems, right next to each other. The first was to do with ActiveSupport::Subscriber’s notion of #children and how it was tracking them. John Hawthorn explains it, and had already fixed it in Rails 7.1 (we were on 6.1 at the time).

This alone wasn’t the cause our memory leak. But in combination with a change to the Bugsnag Gem to start cleaning URLs Rails breadcrumbs, it manifested in a leak. The Bugsnag change relied on URI to parse and then redact certain sensitive query strings. The URI is rightfully pretty strict about what it considers a valid URI, raising an error when given an invalid URI. Meaning the ActiveSupport::Notifications.subscribe block in the Bugsnag Gem could now raise an error while processing an ActiveSupport::Notifications::Event. And raising that error meant that the parent Event wasn’t getting popped off the Subscriber#event_stack. So the parent Event was sticking around (leaking memory), and it was still holding a reference to the child Event via its #children Array (leaking more memory).

John’s change not only removed the Event#children concept, but completely removed the shared-Array used to track Events. The source of both leaks, gone in a single PR. 👏

🛠️ Fix the problem

John’s fix for newer Rails means this is no longer a problem… on newer Rails versions. But we weren’t there yet. As luck would have it, just days prior Bugsnag had fixed the Bugsnag.cleaner.clean_url method to not raise an error on invalid URIs.

Our fix, in the short term, was to upgrade the Bugsnag Gem to a version that included the “don’t raise an error while trying to clean an invalid URI” change. Longer term we’re upgrading Rails versions.

Upgraded dependency no longer raises, and now the stack is empty.

🤡 It’s me, hi, I’m the problem, it’s me.

(with apologies to Taylor Swift) 🙏

Funny story. It turns out that Bugsnag change was introduced to our code base right around when the first memory spikes occurred. We’d upgraded from v6.26.0 to v6.26.1 to fix a deprecation warning for a different dependency. We (by which I mean me, I, myself) audited this change too. And dismissed it as not being related because… why the heck would redacting part of a URI, something the Bugsnag Gem already did in other places, cause such a bad and sudden memory leak!?!? Well… something-something-something assumptions. 🤦

Cherry-picking Specific Active Support Behavior

Wed, 05 Oct 2022 00:00:00 +0000

The Active Support library has always (or, close enough to always) allowed us to cherry-pick specific extensions/behaviors, to load only strictly needed dependencies. That is, rather than loading the entirety of the library, we can load just the bits and pieces we need. This helps keep the amount of things loaded in memory smaller, and faster by doing less work. It can also aide in understanding by making more explicit the dependencies some code relies on.

I’ve been using this technique for years, and it’s been solid. Until today, while working on a Rails 7 code base, when I started seeing an error:

uninitialized constant ActiveSupport::IsolatedExecutionState (NameError)

As it happens, what I’d been doing for years worked, but more by accident than design. Active Support 7 has fixed the glitch.

How it used to “work”

Let’s say we want to use Active Support’s Time extensions for Numeric types. For example, that’s what that allows us to express some past date/time like this:

5.days.ago #=> 2022-09-30 09:08:47.983955 -0400

We could cherry-pick in just those extensions, right in the file where we’re using them, like so:

require "active_support/core_ext/numeric/time"

5.days.ago #=> 2022-09-30 09:08:47.983955 -0400

But as of Active Support 7, that doesn’t quite work. It raises a NameError because the ActiveSupport::IsolatedExecutionState constant cannot be found.

require "active_support/core_ext/numeric/time"

5.days.ago #=>
# NameError: uninitialized constant ActiveSupport::IsolatedExecutionState
# 
#       ::ActiveSupport::IsolatedExecutionState[:time_zone] || zone_default
#                      ^^^^^^^^^^^^^^^^^^^^^^^^
# from .../ruby/gems/3.1.0/gems/activesupport-7.0.4/lib/active_support/core_ext/time/zones.rb:15:in `zone'

What changed?

Active Support 7 added a new abstraction to better handle different concurrency mechanisms (think, Threads vs Fibers). In doing so, everywhere in Active Support that used to directly reference Thread.current, was changed to leverage ::ActiveSupport::IsolatedExecutionState.

That means attempting to directly cherry-pick a bit of Active Support, as had been possible for years, we might also need the IsolatedExecutionState constant loaded. So, the direct cherry-picking feature is… broken?

Not exactly. It turns out, Active Support never intended us to directly cherry-pick by only requiring the one file we need. The intent, and guides, have long told us to first load the library itself, and then cherry-pick any specific bits we want. That is, we should have been doing this all along:

require "active_support"
require "active_support/core_ext/numeric/time"

5.days.ago #=> 2022-09-30 09:08:47.983955 -0400

So really, nothing changed. Before, things just kinda worked by accident.

Does that mean all of Active Support is loaded?

I’d always assumed that requiring the library would also load the whole library. Though, I should have known better. mea culpa.

Instead, requiring the library via require "active_support" only loads the bare minimum, while setting up all of the Ruby autoload-ing needed to reference other bits of the library later. Huzzah!

If we do want or need to eagerly load the whole library, that’s done like so:

require "active_support/all"

Phew, glad to have gotten to the bottom of that one! 🎉

And a shout-out to @gektin for helping me dive into this one.

A Rails 7 compatible bin/dev for heroku local

Thu, 01 Sep 2022 00:00:00 +0000

Rails 7 introduced a new bin/dev wrapper to launch and manage your Rails server, CSS watcher, and JS bundler into a single process, managed by foreman. This is quite handy for running everything with a single command. But what if you’re deploying to Heroku and using the Heroku CLI’s heroku local to run things locally? Or if you’re a fan of one of the other tools that manage processes based on your Procfile?

I’ve got you covered!

`bin/dev` for `heroku local`

The biggest, and most obvious change is swapping out the executable. Mostly this is to do with heroku local using the :<subcommand> style CLI interface. Let’s compare foreman, heroku local, and bin/dev doing similar things to see how their CLIs differ.

# start the web and css processes
$ foreman start web,css
$ heroku local:start web,css
$ bin/dev web,css

It turns out bin/dev explicitly calls the foreman start command. So a quick heroku local version of the same would be:

#!/usr/bin/env sh

if ! command -v heroku &> /dev/null; then
  printf "You must install the Heroku CLI on your system before setup can continue\n"
  printf "☁️  https://devcenter.heroku.com/articles/heroku-cli\n"
  exit 1
fi

exec heroku local:start "$@"

And this works… fine-ish. I mean, it totally works. But it’s only using a small part of the underlying heroku local CLI. And the same goes for the foreman version. If we want to leverage the run or version commands, we’re out of luck.

A more robust `bin/dev`

If we want to leverage the other CLI commands, we’re going to need to do some bash-ing to check and manipulate the passed arguments. Let’s start with the CLI we want to have.

$ bin/dev web,css         # (implicit) start the web and css processes
$ bin/dev start web,css   # (explicit) start the web and css processes
$ bin/dev run bin/rails c # start a rails console session
$ bin/dev version         # check the foreman version

For this we need to default to the start command, but allow overriding it. And we’ll need to pass other arguments on to heroku local.

#!/usr/bin/env bash

if ! command -v heroku &> /dev/null; then
  printf "You must install the Heroku CLI on your system before setup can continue\n"
  printf "☁️  https://devcenter.heroku.com/articles/heroku-cli\n"
  exit 1
fi

cmd='start'
heroku_cmds=('run' 'start' 'version')

# If we were given any positional args we might need to handle sub-commands
if [[ $# -gt 0 ]];then
  first_arg=${1}

  # Shift heroku local commands off the array so we can pass remaining arguments along
  if [[ " ${heroku_cmds[*]} " == *" ${first_arg} "* ]]; then
    cmd="${first_arg}"
    shift
  fi
fi

exec heroku "local:${cmd}" "$@"

🎉 And just like that, it works! We’ve got a bin/dev that works with heroku local and supports its other commands. You can even bin/dev --help to see heroku local’s CLI help text.

I’ll leave it as an exercise to the reader to adapt this for node-foreman or other tools.

Verbose Shell Scripts for Future You

Mon, 01 Feb 2021 00:00:00 +0000

Command line tools and shell builtins can offer both a short- and long-form way to specify options. Sometimes these are also called short and long flags, respectively. The upshot is a user experience with two ways to specify the same option. A short and terse way. And a longer, more verbose way.

$ curl -HLsS http://stevenharman.net
# ⬆ is the same as ⬇
$ curl --head --location --silent --show-error http://stevenharman.net

Both ways are useful, neither right nor wrong, and each with their own trade offs to consider.

Why Both?

Why are there two ways to say the same thing?

It’s a good question. One I’ve asked too. But I’ve not found a definitive answer. Perhaps the truth is lost to history at this point?

From what I can tell, originally (i.e., the early/original *nix flavored OSes) there were only single-character options. At some point multi-character options were added, possibly via GNU.

Along the way the sheer number of options also grew, a lot. Is this a cause and effect relationship, a correlation, or pure happenstance? I don’t know. But today we have A LOT of options, and often multiple ways to express them.

Let’s talk about the trade offs and when to use each style.

Short Options are Good.

They’re more terse. Literally, they are less to type, less to read, and smaller.

These can be desirable traits when writing a one-off commands in your shell. With less, to type you can execute the command sooner. Seems Good™️.

A drawback of this terseness is a decrease in readability and increase in cognitive load. That is to say, they are more difficult for a human (or at least this human) to parse and internally translate into a meaningful “name.” Consider again the example I opened with:

$ curl -HLsS http://stevenharman.net
# ⬆ is the same as ⬇
$ curl --head --location --silent --show-error http://stevenharman.net

To my eyes and brain, parsing and understanding that first line is difficult, at best. Examples like the casing of s vs S referring to different, but related options, abound. It’s also difficult to notice the difference in casing when scanning the code. Especially days, weeks, or years later, when you’ve totally forgotten that there is a difference.

But hey, if I’m just ripping off a quick cURL to check if my site is up, who cares? Presumably I know and remember what each of those -HLsS options are doing, so I saved myself a bunch of typing. Which for me also means a bunch of typos. 🎉

In summary, short options less to write and read, at the expense of not being as obvious or self-describing.

Long Options are Good, Too!

Well, they’re longer, no doubt about that!

I find that makes them easier to read at a glance. And their names help to tell what they do - why they’re being used, in the context. Both properties are immensely valuable when used in long-lived scripts. For example, when as a part of a shell script that my team uses for daily operations. Or perhaps even worse, a script we use rarely, but which does some crucial task.

The glance-ability and memorability of the longer names also pays off when searching for a one-liner after the fact. You might remember an option name and use it as a keyword to reverse fuzzy search (^R) your command history. A trick I use ALL THE TIME!

Above I search curl data and quickly find a prior cURL command that posted some JSON data. Exactly what I needed (because I can never remember exactly how that works). The same goes for the many CLIs and their innumerable options that I use on a daily basis.

A drawback to the verbosity is there’s more to type! Which I suppose leads to more typos and non-zero exit codes. Though, if we’re optimizing for read-time, rather than write-time, maybe that isn’t such a big deal. 🤷

Which Style to Use, and When?

Whatever history might have to say about why we have long options, I prefer to use them when they’re available. That goes for both one-off commands in the shell, and doubly so for scripts that will exist beyond their initial authoring. Admittedly, sometimes I am a bit lazy, or overly rely on muscle memory, and fall back to short options. 🙈

If you don’t already, I’d encourage you to try using long options, especially for scripts. But like I said, neither is right nor wrong. It Depends™️. Consider the trade offs, and move forward.

Well-behaved Ruby Objects: Equality

Tue, 14 Apr 2020 16:40:01 +0000

A “well-behaved” object in Ruby needs to understand the following:

What makes two Ruby objects “equal”?
And which version of “equal” (there are several in Ruby)?
And what makes an object usable as a Hash key?
And is that the same thing that makes them Comparable?

Because I can never seem to remember the specifics. And because my searching the Interwebs seems to find related, but not specific help. And because I’ve got this blurgh-thing… I might as well use it to help future-me (and maybe you?).

Gimme the gist

Implement hash and eql? for use as a Hash key, and then alias the eql? method to == for the expected developer ergonomics. Something like this:

class Message
  attr_reader :body, :subject

  def initialize(subject:, body:)
    @subject = subject
    @body = body
  end

  def eql?(other)
    other.class == self.class &&
      other.body == body &&
      other.subject == subject
  end
  alias == eql?

  def hash
    [self.class, body, subject].hash
  end
end

What to know more about the specifics, or how to also make these objects Comparable? Read on, friend…

Surely someone else has covered this already

As it turns out, Russ Olsen covers this very topic, in depth and with great examples, in his book, Eloquent Ruby. I should know that because I’ve read that book, and recommend it! However, it always takes me several trips to The Googles to find a snippet which eventually points me back to that book. At which point I have to dig out my copy. All of this only after I’ve tried a dozen other search terms resulting in articles about the difference between the many forms of equality in Ruby.

So yes, to various degrees, it’s been covered before. There is nothing new under the Sun.

Like I said, I have this here blurgh and I’m gonna use it. 🤷

Every-day object equality

For every-day use, the form of object equality we most often use in Ruby is the ==, or double-equals, form. By default, the == operator is an alias of the equal? method, which it inherits from Object. This form compares the object identity, effectively asking “are these the same object in memory?”

We don’t want this form, so we need to override it to something more meaningful for our object. Often two objects are ==, or double-equals equal, if they are of the same class, and some set of their internal data is also ==. In the case above, that means both objects are a Message, and their body and subject are the same.

def ==(other)
  other.class == self.class &&
    other.body == body &&
    other.subject == subject
end

Please go read Mr. Olsen’s excellent coverage of the topic for more details and in-depth examples.

Use as a `Hash` key

When used as a Hash key, Hash will call the object’s hash method to get hash code (an Integer value) to use when storing and searching for the key. It then uses the object’s eql? method to determine if two hash keys are the same key. From the Ruby docs:

hash → integer

Generates an Integer hash value for this object. This function must have the property that a.eql?(b) implies a.hash == b.hash.

The hash value is used along with eql? by the Hash class to determine if two objects reference the same hash key. Any hash value that exceeds the capacity of an Integer will be truncated before being used.

The hash value for an object may not be identical across invocations or implementations of Ruby. If you need a stable identifier across Ruby invocations and implementations you will need to generate one with a custom method.

By default, objects inherit their hash and eql? methods from Object, which compares the object identity, similar to equal?. As you might guess, we don’t want that for our object.

For hash, we can lean on Ruby already knowing how to calculate the hash code for an Array of values. We’ll use all of the values composing the identity of the object itself.

def hash
  [self.class, body, subject].hash
end

To determine if two objects are eql?, and hence, the same keys in a hash, we want to make sure they’re of the same class, and some set of their internal data is also ==.

def eql?(other)
  other.class == self.class &&
    other.body == body &&
    other.subject == subject
end

Look familiar? It’s the same as the == from above!

In practice, I often implement eql? and hash since they are a bit of a duo - one requiring the other. And then I alias == to be the same as eql?

def eql?(other)
  other.class == self.class &&
    other.body == body &&
    other.subject == subject
end
alias == eql?

And that’s it, these objects now know if they are everyday-equal, and can serve as Hash keys. We’ve not covered === here; the links above provide explanations about when, why, and how you might need to reach for that form of equality amongst Ruby objects. Again, go check out Mr. Olsen’s book; it’s great!

What about `Comparable`?

Being able to compare, and thus sort two Ruby objects is a bit tangential to the forms of equality we’ve discussed here.

Objects with a “natural ordering” will implement the <=> or spaceship operator. This operator is used to determine if one object is less than the other, greater than the other, or equal to the other. Ruby’s Comparable module will use that to implement several comparison operators. And all of that can be use by other Ruby modules, like Enmerable to do things like sort a collection.

🤓 Neat!

Again, Mr. Olsen’s book, or any number of other articles, including the Ruby docs for Comparable, cover this in great detail.

The gist, once more

To recap, implement the hash and eql? methods to use an object as a Hash key. Then alias the eql? method to == to cover they typical, “every day”, usage we expect when writing/reading Ruby.

Debugging Homebrew with Pry

Fri, 17 Jan 2020 21:00:01 +0000

Over the years I’ve written a few Homebrew formulas and sent the occasional Pull Request to update a formula or two. But I’ve never done any work within Homebrew. I’ve never needed to debug how Homebrew itself worked. Until now.

I assumed our typical Ruby debugging tools, like Pry and Pry-Byebug would work. Homebrew is just Ruby, after all. Which is true. But it’s also a bit special, and we can’t do the normal require "pry-byebug"; binding.pry tricks.

But we can, with a little poking around, still use those tools!

A Homebrew command

In this case, I was exploring a built-in command to see if I could extend it. The brew-postgres-upgrade-database command is a great little tool that ✨ automagically ✨ upgrades a PostgreSQL instance’s data between major versions.

$ brew postgresql-upgrade-database
==> Upgrading postgresql data from 11 to 12...
==> Moving postgresql data from /usr/local/var/postgres to /usr/local/var/postgres.old...
==> Creating database...
==> Migrating and upgrading data...
==> Upgraded postgresql data from 11 to 12!
==> Your postgresql 11 data remains at /usr/local/var/postgres.old

Except it only worked with the default PGDATA location. Which is sensible. But for… reasons, I had my PGDATA directory in a different location. I keep mine within my user directory at ~/.pg-data.

A first, failed attempt

The first step was figuring out where this command lived on my machine so I could hack at it, as a proof-of-concept. I needed it to allow an optional data directory.

$ brew --prefix
/usr/local

Knowing Homebrew was installed in /usr/local I went there and poked around until I found what I was looking for in /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/cmd/brew-postgresql-upgrade-database.rb. Phew, that’s a mouthful!

From there I opened the code.

$ vim /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/cmd/brew-postgresql-upgrade-database.rb

Right above the existing code (snipped below for brevity) I tried the normal trick.

 #:  * `postgresql-upgrade-database`:
 #:    Upgrades the database for the `postgresql` formula.
+ 
+require "pry-byebug"; binding.pry
 
 name = "postgresql"
 pg = Formula[name]

Then I tried to run the command, hoping to pop a Pry session.

$ brew postgresql-upgrade-database
Error: cannot load such file -- pry-byebug
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/cmd/brew-postgresql-upgrade-database.rb:6:in `<top (required)>'
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
/usr/local/Homebrew/Library/Homebrew/vendor/portable-ruby/2.6.3/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
/usr/local/Homebrew/Library/Homebrew/utils.rb:82:in `require?'
/usr/local/Homebrew/Library/Homebrew/brew.rb:108:in `<main>'

😔 Well, crap.

A second, successful attempt!

On a lark, I searched the Homebrew code base for references to Pry, hoping for an Issue or someone else talking about this. And it turns out, Homebrew has some Pry support built it.

 #:  * `postgresql-upgrade-database`:
 #:    Upgrades the database for the `postgresql` formula.
+ 
+ Homebrew.install_gem_setup_path! "pry"
+ Homebrew.install_gem_setup_path! "pry-byebug", executable: "pry"
+ require "pry-byebug"; binding.pry
 
 name = "postgresql"
 pg = Formula[name]

Leveraging that prior work I was able to get a full Pry Byebug session to pop.

$ brew postgresql-upgrade-database

From: /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/cmd/brew-postgresql-upgrade-database.rb @ line 10 :

#:                    files. Same as the pg_ctl -D flag.
Homebrew.install_gem_setup_path! "pry"
Homebrew.install_gem_setup_path! "pry-byebug", executable: "pry"
require "pry-byebug"; binding.pry

 => 10: name = "postgresql"
pg = Formula[name]
bin = pg.bin
var = pg.var
version = pg.version
data_dir = options[:data_dir] || var/"postgres"

🎉 Huzzah!

And then, a Pull Request

From there it was a matter of stepping through, line by line, to understand how the command worked. Then some hackity-hacking to allow an optional data directory to be passed in. The net result was a Pull Request allowing you to upgrade a PostgreSQL database with a custom data directory.

$ brew postgres-upgrade-database --D ~/.pg-data
==> Upgrading postgresql data from 11 to 12...
==> Moving postgresql data from ~/.pg-data to ~/.pg-data.old...
==> Creating database...
==> Migrating and upgrading data...
==> Upgraded postgresql data from 11 to 12!
==> Your postgresql 11 data remains at ~/.pg-data.old

🤞 Fingers crossed that PR is merged.

Either way we (or at least I) learned how to debug Homebrew itself. Neat.

In Search Of… Enumerable#transform for Ruby

Fri, 16 Nov 2018 15:56:01 +0000

In Search Of… a Ruby method with the semantics of Enumerable#map and Enumerable#inject. That is, to transform values while also having access to the prior iteration return value. Sounds odd, I know.

Given the following “value object”

class Snapshot
  def initialize(value = 0)
    @value = value
  end

  def apply(new_value)
    self.class.new(@value + new_value)
  end
end

I want to build an array of snapshots, based on some set of inputs. Meaning, I think I’d like something that looks like this

(1..10).transform(Snapshot.new) { |prev_snap, i|
  prev_snap.apply(i)
}

Resulting in this

#=> [#<Snapshot:0x00007fe550d12558 @value=1>,
#=>  #<Snapshot:0x00007fe550d12508 @value=3>,
#=>  #<Snapshot:0x00007fe550d124e0 @value=6>,
#=>  #<Snapshot:0x00007fe550d124b8 @value=10>,
#=>  #<Snapshot:0x00007fe550d12468 @value=15>,
#=>  #<Snapshot:0x00007fe550d12440 @value=21>,
#=>  #<Snapshot:0x00007fe550d12418 @value=28>,
#=>  #<Snapshot:0x00007fe550d123c8 @value=36>,
#=>  #<Snapshot:0x00007fe550d123a0 @value=45>,
#=>  #<Snapshot:0x00007fe550d12378 @value=55>]

Right now, as of Ruby 2.5, I don’t know of a clean way of doing that. Clean being in the eye of the beholder, I suppose. It can be done with Enumerable#map or Enumerable#inject, but it ain’t pretty.

`Enumerable#map`

Here we set an initial value, and re-assign it in each iteration.

snap = Snapshot.new
(1..10).map { |i| snap = snap.apply(i) }

Which works, but setting up the initial value only so we can re-assign it a new value each iteration feels odd.

`Enumerable#inject`

In this case we use an awkward initial setup (pre-calculating the first iteration), and juggle the real value in the block.

🎩 Hat tip to Matt Jones for this one.

(2..10).inject([Snapshot.new(1)]) { |snaps, i|
  snaps.push(snaps.last.apply(i))
}

On the up side, both options product the following, desired output!

#=> [#<Snapshot:0x00007fe550d12558 @value=1>,
#=>  #<Snapshot:0x00007fe550d12508 @value=3>,
#=>  #<Snapshot:0x00007fe550d124e0 @value=6>,
#=>  #<Snapshot:0x00007fe550d124b8 @value=10>,
#=>  #<Snapshot:0x00007fe550d12468 @value=15>,
#=>  #<Snapshot:0x00007fe550d12440 @value=21>,
#=>  #<Snapshot:0x00007fe550d12418 @value=28>,
#=>  #<Snapshot:0x00007fe550d123c8 @value=36>,
#=>  #<Snapshot:0x00007fe550d123a0 @value=45>,
#=>  #<Snapshot:0x00007fe550d12378 @value=55>]

`Enumerable#transform`, maybe? 🤔

So, how about that Enumerable#transform I mentioned? Well, it’s completely made up. By me. Just now.

(1..10).transform(Snapshot.new) { |prev_snap, i|
  prev_snap.apply(i)
}

I’m not particularly sold on the name #transform, nor the other particulars. Perhaps a new method isn’t even needed? An optional argument to #map might be sufficient?

(1..10).map(Snapshot.new) { |i, prev_snap|
  prev_snap.apply(i)
}

This would mean there’d need to be a default initial value, of course. nil, perhaps? Or maybe follow #inject’s lead and if no initial value is provided, use the first value of the collection?

If you do not explicitly specify an initial value for memo, then the first element of collection is used as the initial value of memo. Ruby Docs Enumerable#inject

Or… maybe not? 🤷

I’m not sure if this is needed. Nor how useful it would be.

What I do know is I reached for something like this today, and to my surprise, it wasn’t there. I’ve become so accustomed to Ruby (sometimes with the aide of ActiveSupport) having just the ditty I need, that I’m surprised when it doesn’t.

The goal of Ruby is to make programmers happy. I started out to make a programming language that would make me happy, and as a side effect it’s made many, many programmers happy.

I hope to see Ruby help every programmer in the world to be productive, and to enjoy programming, and to be happy. That is the primary purpose of Ruby language. Yukihiro “Matz” Matsumoto

What say you? A potential addition for Ruby vNext? Or am I expecting too much programmer happiness from this great language?

Reclaim Your Domain Model from Rails

Fri, 28 Feb 2014 19:51:47 +0000

TL,DR; When building an application using Rails, I prefer to keep all my model in app/models/. I reserve lib/ for those other things - those not-my-domain-things. I’d like to explain the what and why.

Rails has a history of co-opting names, as happened when the ActiveRecord library used the active record pattern name. A similar co-opting has happened with the MVC pattern wherein many believe Rails is an example of the MVC design pattern. In truth, it’s probably closer to MVC Model 2… but I digress.

Model View What’s-that-now?

MVC stands for Model, View, Controller. In Rails-land we know what the Controllers are. And while we don’t have Views in the way that MVC meant, we do have view-templates, and we call those our views. The Model is meant to be all the things it takes to model our problem domain. As applied to Rails, the Model seems the most misunderstood/misused of the MVC triumvirate.

Rails gave us the app/models/ directory, a natural home for our model. Yet, that was also where Rails dumped all our ActiveRecord::Base-derived objects. Combined with the “Skinny Controller, Fat Model” mantra of 2006-ish Rails-land, that directory had grown full of bloated, tightly coupled, low cohesion ActiveRecored::Base-derived objects. Our model!

Models Too Fat?

Eventually we realized that shoving all those disparate concerns into a single class was a Bad Idea™. There was a movement afoot to thin down our now overly-fat models. So, we re-learned some forgotten lessons and the use of POROs became a Good Thing™. We needed to break our domain models down into smaller, more cohesive objects, but we had a new problem. Where to put these new files?

As Rails had reserved app/models/ directory for ActiveRecord::Base-derived objects, we had to find a new home for those POROs. As luck would have it, Rails had given us a lib/ directory, and that thing was basically deserted. We began breaking up our model and littering it amongst the app/models/ and lib/ directories.

Our model had become more loosely coupled, with more cohesive parts, but it wasn’t living together. Why are we forcibly segregating the ActiveRecord::Base portions from the rest of our model? If it’s a single domain model, why doesn’t it live together?

These days I put all the domain-specific objects into the app/models/ directory. I reserve the lib/ directory for things which could ostensibly be gemified.

So. Many. Files. So Many Boundaries.

I’ve heard grumblings that having all the domain model living together results in a lot of files in the app/models/ directory, making it hard to organize and navigate. To this I say, YES, it does.

However, I suspect that is a sign of another problem. To wit, do you actually have multiple domains emerging, which could perhaps be broken out into discrete apps? If that is the case, make that boundary explicit and break out a new application!

So long as those other domains are small or emerging it seems excessive to split them out into their own apps. So, what do we do? Why we segregate them with internal domain boundaries, of course!

An Example Boundary.

A lot of apps have a search feature (or set of search features). Search is a whole domain on its own, and often not germane to the primary domain. Pull all search concerns into a namespace, Search::. To help with organization, put all files to do with search in a directory of their own. For example, app/models/search/, app/controllers/search/, app/views/search/, etc…

With just a few changes we’ve reclaimed the app/models/ directory, better organized our files, and defined some new boundaries within our code base. Ah, much better.

Deep Greens image courtesy of Brett Jordan.