Learnings

I decided recently that it'd be valuable to keep a running list of things that I've learned in the web dev world (or software engineering more broadly). I took inspiration from Meagan Dunham who posts an update to a thread about something they've learned every time they figure something new out. It's great, so I figured I'd emulate a bit by making this a featured post that I update periodically.

2019.04.10

1. CI/CD permission issues

When you're putting a shell script into your CI/CD, make sure it is executable . chmod +x <script-name>.sh.

2019.04.07

1. Liquibase lock file

When you send the kill signal while running a liquibase update script, it may not update the databasechangeloglock table. This is what an example might look like:

=# select * from databasechangeloglock limit 1;

 id | locked | lockgranted | lockedby 
----+--------+-------------+----------
  1 | f      | [NULL]      | [NULL]
(1 row)

In my case, my script would fail immediately because the locked was set to true, and lockedby was set to my IP address (liquibase must do this under the hood).

2019.01.30

1. Liquibase changeset importance

I just got started recently working w/ liquibase (but that's a story for a longer post), and I had a bit of a mismatch.

On my local db, I had played around and created a table that didn't exist in other environments. I wrote a new changelog sql changeset like so:

--liquibase formatted sql

--changeset drewkakes:1 runOnChange:true
CREATE TABLE IF NOT EXISTS drone ()
-- ...

And so I ran my delta, and was surprised when I saw that it wasn't updating my table. I remembered that maybe I need to drop it first, so after running drop table drone; and trying again, it didn't create the table at all. I was baffled.

What I forgot, is that every time you run an update with liquibase, it adds to the databasechanges table with the metadata. It had already read the --changeset drewkakes:1 and since it knew it'd already run, it wouldn't re-run that delta. (unless I set some other parameters.

After realizing that, the change was as simple as changing to --changeset drewkakes:2 and lo and behold it ran correctly, knowing that it was a new change to be added.

2018.12.18

1. Scala/Scalatra and default returns

In JavaScript, I prefer to do things like so:

if ( badState ) {
  return "400";
}

// do logic
return "OK";

In Scala, however,

The last expression in the body is the method’s return value. (Scala does have a return keyword, but it’s rarely used.) Scala Basics

So when I tried the following:

if ( badState ) {
    "400"
}

// do logic
"OK"

I expected to get "400" with a bad request, but instead was getting "OK" in my response. To fix this returning-early situation, I just needed to wrap the rest of the logic in an else block. This made the "400" the last expression in the body, so it would return. All good :).

2018.12.13

1. PostgreSQL function modification

Situation required me to modify a function that we currently use. call it fn_get_hierarchy. We use Liquibase for doing incremental changes to our database through deltas, and allows us to keep it under version control. Essentially the functions looked like this:

-- old function
DROP FUNCTION IF EXISTS fn_get_hierarchy(text, text [], text [], text, integer, integer);
CREATE OR REPLACE FUNCTION fn_get_hierarchy(
  _parent_org_sk text,
  _org_sks       text [],
  _section_sks   text [],
  _role_code     text,
  _limit         integer DEFAULT 10000000,
  _offset        integer DEFAULT 0)
  
-- new function
DROP FUNCTION IF EXISTS fn_get_hierarchy(text, text [], text [], text, text, integer, integer);

CREATE
OR REPLACE FUNCTION fn_get_hierarchy(
  _parent_org_sk text,
  _org_sks       text [],
  _section_sks   text [],
  _role_code     text,
  _program_id    text DEFAULT 'ND',
  _limit         integer DEFAULT 10000000,
  _offset        integer DEFAULT 0)

There were two problems here:

1. My replacement code did not specify the right signature for destroying the old function
2. My replacement code only added a new optional parameter.

As such, when/if the functions were called like this: SELECT * from fn_get_hierarchy('string1', Array['string2'], ARRAY['string3'], 'string4') it would actually match both of the signatures, resulting in PostgreSQL not knowing which one to actually use, and spitting out this ugly error:

ERROR: function fn_get_hierarchy(character varying, character varying[], character varying[], character varying) is not uniqueHint: Could not choose a best candidate function. You might need to add explicit type casts.

So in the future make sure to drop your old signature functions, and if you add defaults, make sure it won't match another signature of the same name.

2018.11.20

1. File Reading & type width in C

I was working through CS50's problem set related to recovering JPEG data from a borked .raw file. I was looking through the blocks like this:

FILE *file = fopen("filename", "r");

// allocate a block of length 512 (one jpg unit/thing)
int block[512];
fread(&block, 512, 1, *file);

printf("I expect a value here that is one byte: %i", block[0]);	
// actually yielded 0xffd8ffe0

Wait a second, I asked for 1 byte and you gave me 4. The problem here is that an int is actually 32 bits (4 bytes). Check out this C Reference for deets.

The 32-bit int data type can hold integer values in the range of -2,147,483,648 to 2,147,483,647.  You may also refer to this data type as signed int or signed.  

Solution was to use something smaller like unsigned char. voila.

2018.11.16

1. Reverse DNS

Recently I was tasked with learning Scala in order to work on some of our back-end work at McGraw-Hill. Someone mentioned "Reverse-DNS" in reference to why the module we were working on was nested 30 bajillion folders deep. I was only used to hearing DNS referred to in the web, so hearing it as a naming convention was odd to me. So essentially it looks like this:

src
├── main
│   ├── resources
│   │   └── datafiles
│   │       └── lookups
│   │           └── v1
│   ├── scala
│   │   └── com
│   │       └── mhe
│   │           └── measures
│   │               └── delivery
│   │                   ├── avalon
│   │                   │   ├── dataservice
│   │                   │   ├── db
│   │                   │   └── schema
│   │                   ├── dataservice
│   │                   │   ├── habitat
│   │                   │   └── lookups

Damn that's long and confusing, but I understand the value of avoiding naming collisions, so I guess we're even?

2. Getting a Postgres version using a query

SELECT version();

Returns the detailed version from whatever host you're connected to :).