id | received_at |
---|---|
DA588117-0F6A-41AA-B0B0-D0DA20F21745 | 2022-10-04 19:51:04.271+00 |
4ca41477-a9ed-4ea2-a357-7ba5c19cc27c | 2022-10-04 19:46:49.421+00 |
CDB9B507-44BC-4BEE-B9D8-3D230C59FF01 | 2022-10-04 19:45:45.545+00 |
783236a1-1a50-4191-a4f6-fceca77fc991 | 2022-10-04 19:45:41.645+00 |
BC6C1BC0-02D4-461A-9EDD-EBF98BF47574 | 2022-10-04 19:45:28.542+00 |
43F3C421-6CA9-40E0-AA79-AD36A85D5EF5 | 2022-10-04 19:45:19.204+00 |
F45C4B33-40CC-422F-B5F5-1982155656F7 | 2022-10-04 19:44:59.033+00 |
890b2d6b-6383-4af9-a556-ab9480ffc963 | 2022-10-04 19:44:54.164+00 |
7570242a-c646-4224-836a-73637e7bbeeb | 2022-10-04 19:44:54.164+00 |
26a14851-daee-4aee-b13d-08b06ea24287 | 2022-10-04 19:44:54.164+00 |
DE32C925-1528-4008-A404-DD38307C6138 | 2022-10-04 19:44:38.161+00 |
Querying this table was slow, resulting in dashboard views that took 60+ seconds to load, and I wanted a quick win to reduce the size of it. Interestingly, the casing wasn’t consistent, but that shouldn’t matter.
CREATE TABLE tmp_raw_table
SELECT
id as event_id,
received_at
FROM segment.event
LIMIT 100000;
CREATE TABLE tmp_raw_uuid
SELECT
id::uuid as event_id,
received_at
FROM segment.event
LIMIT 10000000;
SELECT pg_size_pretty( pg_total_relation_size('tmp_raw_table') );
SELECT pg_size_pretty( pg_total_relation_size('tmp_uuid_table') );
Running this to test my syntax, and the reported size of the differences in tables goes from 7488 kB to 5096 kB. Not bad for something as low-effort as coercing a type, so I tried to ship it to the full table, and then it failed…
ERROR: invalid input syntax for type uuid: "0qREGUQhbFNoRd-aeFacw"
In my experience, halting functions like these are nasty when used in a production ETL flow. Imagine if a malformed ID like that were introduced months later; now the flow is broken until developer time can be allocated to fix it. Gross. In some situations you want your code to fail-fast, but I just wanted a damned UUID. Or maybe null would have been acceptable. This post had a good snippet, so I turned that into a database function.
CREATE OR REPLACE FUNCTION to_uuid(raw text)
RETURNS uuid IMMUTABLE STRICT
AS $$
BEGIN
RETURN raw::uuid;
EXCEPTION WHEN invalid_text_representation THEN
RETURN uuid_in(overlay(overlay(md5(raw) placing '4' from 13) placing '8' from 17)::cstring);
END;
$$ LANGUAGE plpgsql;
There are a lot of good reasons to avoid database functions, but I think as long as they’re very concise, it shouldn’t be any more annoying than using built-in functions. I added that IMMUTABLE
bit to make sure it’s deterministic and fast/safe to use in indexes.
SELECT
to_uuid(id) as event_id,
received_at
FROM segment.event;
For a given user, their Lness classifier is defined as the number of days in the last n-days in which they engaged with the app.
If n is 7 (the last week), and a user opened the app 5 times on Monday, 10 times on Wednesday, and 1 time on Saturday, then their Lness is a 3. The amount of activity on a given day does not matter, so long as it’s non-zero. The Lness will be an integer between 0 to n.
Engage is ambiguous, and depends on what your app’s key activities are; is it that they open the app? watch a video? like someone’s status update? In the case below, I have an event stream table, which could come from Segment or Amplitude or something home grown, and I define engagement as any user-driven action. If you have offline events in this table (such as an events fired from scheduled drip campaigns), you’ll want to exclude these.
To calculate this with an SQL query going to Postgres, I took advantage of the ISO Week Date formatting built in, otherwise the week that wraps around the year marker would break up my metrics.
To calculate this, my first attempt looked something like this:
SELECT
e.user_id,
COUNT(DISTINCT EXTRACT(DAYS FROM CURRENT_TIMESTAMP - e.sent_at))
FROM event e
WHERE e.sent_at > CURRENT_TIMESTAMP - INTERVAL '7 days'
GROUP BY e.user_id
Which says for every user, give me all of the events from the last 7 days, and then extract the number of days old each event is, and then count the number of distinct number of days there are. But this was a mistake because I misunderstood two things:
Understanding these two things, my next query looked like this:
SELECT
e.user_id
TO_DATE(TO_CHAR(e.sent_at, 'IYYY-"W"IW'), 'IYYY-"W"IW') AS iso_week,
COUNT(DISTINCT TO_CHAR(e.sent_at, 'ID')) AS lness
FROM event e
WHERE e.user_id IS NOT NULL
GROUP BY 1, 2
Which says _for every combination of user_id
and iso-week from the sent_at
(forget about the day of week and time), convert the sent_at
to just the day of the week, and tell me how many distinct days of week there are for every combination of the first two things I queried.
user_id | iso_weeek | lness |
---|---|---|
40232 | 2021-09-19 | 1 |
40232 | 2021-09-26 | 3 |
40232 | 2021-10-03 | 2 |
42781 | 2021-09-26 | 1 |
41301 | 2021-09-26 | 7 |
41301 | 2021-10-03 | 6 |
… | … | … |
The data is now in a useful proper cardinality and schema for presentation. I prefer Superset because it’s free, although Preset is a very affordable hosted alternative. The above query will probably take a while to run (it took me about 25 seconds), and Preset will attempt to cache data returned. With a Time-series Bar Chart configured as shown on the left, it shows me a graph like I had envisioned.
]]>const f = () => console.log("f() called");
f();
// f() called
f();
// f() called
f();
// f() called
But that’s so basic. You can also do it this way.
[1, 2, 3].forEach(f);
// f() called
// f() called
// f() called
// => undefined
But those constant literals there are clouding up the whole business. Gross, but we can remove them!
[, ,].forEach(f);
// => undefined
But wait! f()
isn’t called for every element, then! What gives? There’s a hint here if we use Array.map
instead…
[, ,].map(f);
// => [ <2 empty items> ]
It’s because of copy elision here. But if we use Ramda#map, it does work.
const R = require("ramda");
R.map(f, [, ,]);
// f() called
// f() called
// => [ undefined, undefined ]
But it’s only called twice! What gives? This is because of trailing commas in arrays. It’s a feature, not a bug.
const R = require("ramda");
const ff = R.map(f);
ff([, , ,]);
// f() called
// f() called
// f() called
// => [ undefined, undefined, undefined ]
Enjoy your job security!
]]>Computer Typewriter
So I reinvented the wheel. I got the OTF fonts from CTAN, which are usable as they are but I compressed them to WOFF2 with this, so they’re only half the size. Now I can include them with the CSS:
<link rel="stylesheet" href="https://philihp.com/assets/fonts/fonts.css" />
<style>
body {
font-family: "Computer Modern", serif;
font-display: fallback;
}
pre,
code {
font-family: "Computer Typewriter", monospace;
font-display: fallback;
}
</style>
You can use this as long as you don’t care for anyone using Internet Explorer, and I’m okay with that.
The value you use for font-display
is probably personal taste. How do you feel about late swaps of the font? I think fallback
is a good compromise. IE also fails at this.
Smash the garlic and mince it as fine as you can get, this should take at least 10 minutes to maximize allicin production, which stops once you added to heat. Melt butter in pot on high heat. Add garlic and cook on high until brown but not blackened. Add chicken broth first, then milk, then pasta. Bring to boil and then reduce to a simmer. Stir often to avoid burning milk fats. In a small pan, on high toast peppercorns, then grind and add to pot. After about 20 minutes the pasta wll absorb the water, keep stirring. Once pasta is mostly tender, add cheese. Stir until stirred.
]]>The HKP protocol is really just a standardized interface of GET parameters for the path /pks/lookup
. This is well documented at keys.openpgp.org and per the OpenPGP spec. At a minimum, if you know the ID of the key you’re searching for, you can ask your keyserver for it with
❯ gpg --keyserver hkps://pgp.philihp.com --recv-key 5B640B9F9600F122
HKPS is just HKP over SSL, simillar to HTTP and HTTPS. Since I deplyed this Next.js app via Vercel, it will insist people use HTTPS, so you can’t just do --keyserver pgp.philihp.com
, because this will try to query over HTTP.
Since my keyserver always insists on sending you my key, if you query it with any other key ID, a key will come back and GnuPG will reject it.
❯ gpg --keyserver hkps://pgp.philihp.com --recv-key CE90A31451DE4AD7
gpg: key 0x5B640B9F9600F122: rejected by import screener
gpg: Total number processed: 1
¯\_(ツ)_/¯
The source code is available on Github
]]>keys.openpgp.org
keyserver and there’s little reason not to have it active on your domain to help it gain traction.
Simply set a record for your domain’s DNS named openpgpkey
that CNAME’s to wkd.keys.openpgp.org
That’s it.
For example, I have the email address philihp@theunhatched.com
, and a client can discover my PGP key automatically with a command like
❯ gpg --locate-keys --auto-key-locate wkd philihp@theunhatched.com
With this domain, you can verify that WKD is properly setup with the command
❯ dig -t CNAME openpgpkey.theunhatched.com
...
...
...
;; ANSWER SECTION:
openpgpkey.theunhatched.com. 600 IN CNAME wkd.keys.openpgp.org.
...
...
...
This is an improvement over DNS-based DANE bindings for OpenPGP because it doesn’t require each user to have their own DNS record. That might work for personal domains, but this scales for servers where you host email for users that maybe they shouldn’t have access to edit your DNS tables.
]]>I made a post validating the performance of Openskill to be equivalent to Microsoft TrueSkill. Recently, I’ve played a few games of Twilight Imperium with friends, and came across this wonderful BoardGameGeek thread with almost five thousand self-reported matches of each race and the game result. It’s all self-reported, however intuitively with this many results, any dirty data from bias should come out in the wash. Below are ordinally ranked races.
Race | Rating | μ (Mean) | σ (Stddev) |
---|---|---|---|
Universities of Jol-Nar | 25.17783798 | 27.75971891 | 0.8606269782 |
Federation of Sol | 24.68035069 | 27.20558052 | 0.8417432765 |
Emirates of Hacan | 24.15905166 | 26.70351120 | 0.8481531815 |
Clan of Saar | 23.32228024 | 26.36106674 | 1.0129288330 |
Naalu Collective | 23.15130408 | 26.09800162 | 0.9822325127 |
Yssaril Tribes | 22.59962450 | 25.51847678 | 0.9729507572 |
L1Z1X Mindnet | 22.39603483 | 25.17927526 | 0.9277468094 |
Barony of Letnev | 22.31060757 | 25.07096185 | 0.9201180951 |
Winnu | 21.67067419 | 25.70762296 | 1.3456495890 |
Mentak Coalition | 21.46529532 | 24.31886462 | 0.9511897685 |
Yin Brotherhood | 21.32122119 | 24.60400779 | 1.0942622000 |
Xxcha Kingdom | 20.88291125 | 23.58119643 | 0.8994283962 |
Nekro Virus | 20.11306737 | 23.12521055 | 1.0040477260 |
Embers of Muaat | 19.66918023 | 22.75911323 | 1.0299776660 |
Ghosts of Creuss | 19.64592785 | 22.51871286 | 0.9575950046 |
Arborec | 19.61730334 | 22.61314918 | 0.9986152823 |
Sardakk N’orr | 18.74714691 | 21.89530442 | 1.0493858370 |
Rating here is calculated by μ-3σ, so the algorithm is saying there’s a 99.7% probability that race’s rating is at least what it reports.
Since race abilities are asymmetric, and tend to perform with varying degrees of success dependent on the number of opponents, I broke these down further among 3, 4, 5, and 6 player games. This is interesting to see how each is different vs their respective mean performance. Sample size of each of these is approximately the same.
Race | Rating |
---|---|
Universities of Jol-Nar | 22.72400553 |
Clan of Saar | 22.01709142 |
Naalu Collective | 21.74858024 |
Barony of Letnev | 20.81119467 |
Federation of Sol | 20.68230305 |
Winnu | 20.25990961 |
Yssaril Tribes | 19.81126008 |
Yin Brotherhood | 19.10534831 |
Arborec | 18.61340575 |
Emirates of Hacan | 17.77329450 |
Sardakk N’orr | 17.69178725 |
Mentak Coalition | 17.00070891 |
Xxcha Kingdom | 16.69167889 |
L1Z1X Mindnet | 16.28278617 |
Ghosts of Creuss | 15.57459355 |
Embers of Muaat | 14.98028539 |
Nekro Virus | 14.14631767 |
Race (4 player) | Rating |
---|---|
Universities of Jol-Nar | 23.56908654 |
Emirates of Hacan | 22.81017836 |
Federation of Sol | 22.30073140 |
Barony of Letnev | 21.85553201 |
Naalu Collective | 21.22135540 |
Clan of Saar | 20.88426216 |
Yssaril Tribes | 20.56605905 |
L1Z1X Mindnet | 20.44148444 |
Mentak Coalition | 19.80596262 |
Ghosts of Creuss | 19.63050116 |
Xxcha Kingdom | 19.45644375 |
Arborec | 19.29407333 |
Nekro Virus | 18.67410524 |
Winnu | 18.13440263 |
Yin Brotherhood | 16.88316144 |
Embers of Muaat | 16.14584843 |
Sardakk N’orr | 15.72850978 |
Race (5 player) | Rating |
---|---|
Federation of Sol | 22.60698818 |
L1Z1X Mindnet | 21.73337138 |
Emirates of Hacan | 21.67345473 |
Universities of Jol-Nar | 21.12904255 |
Yssaril Tribes | 20.53781686 |
Yin Brotherhood | 20.03017664 |
Naalu Collective | 19.80487597 |
Clan of Saar | 19.74103594 |
Xxcha Kingdom | 19.11800700 |
Mentak Coalition | 18.63494300 |
Embers of Muaat | 17.89924484 |
Ghosts of Creuss | 17.52308272 |
Nekro Virus | 17.35853234 |
Barony of Letnev | 17.00026925 |
Winnu | 15.66049172 |
Arborec | 14.65157760 |
Sardakk N’orr | 14.20121780 |
Race (6 player) | Rating |
---|---|
Federation of Sol | 24.26156239 |
Universities of Jol-Nar | 24.15462774 |
Emirates of Hacan | 23.78689351 |
Clan of Saar | 21.88890502 |
Naalu Collective | 21.81149146 |
L1Z1X Mindnet | 20.86840867 |
Yssaril Tribes | 20.03274522 |
Barony of Letnev | 19.87474131 |
Mentak Coalition | 19.57421196 |
Winnu | 19.22243110 |
Xxcha Kingdom | 18.19493305 |
Nekro Virus | 18.09263800 |
Embers of Muaat | 17.99331308 |
Yin Brotherhood | 17.82981651 |
Arborec | 16.92892221 |
Sardakk N’orr | 16.45328180 |
Ghosts of Creuss | 15.88655724 |
One takeaway here is Winnu performs better than expected, with a high degree of variance. The race is also vastly unpopular, and played about half as often as other races so Openskill has a harder time narrowing down on a rating. We probably can’t say that experienced players will overestimate them, though. Since the source data contains a self-described “experience” classifier, we can see how this performs with beginners versus intermediate and advanced players.
Race | Skill | Rating |
---|---|---|
Federation of Sol | Advanced | 30.60907345 |
Universities of Jol-Nar | Advanced | 30.00323694 |
Emirates of Hacan | Advanced | 29.32877358 |
Yssaril Tribes | Advanced | 28.29546669 |
Naalu Collective | Advanced | 27.88565763 |
Clan of Saar | Advanced | 26.84950496 |
Mentak Coalition | Advanced | 26.48814197 |
L1Z1X Mindnet | Advanced | 25.98531610 |
Xxcha Kingdom | Advanced | 25.36631530 |
Universities of Jol-Nar | Intermediate | 25.04583848 |
Federation of Sol | Intermediate | 24.99641470 |
Barony of Letnev | Advanced | 24.46872531 |
Emirates of Hacan | Intermediate | 24.45491619 |
Embers of Muaat | Advanced | 24.09007175 |
Yin Brotherhood | Advanced | 23.94127557 |
Barony of Letnev | Intermediate | 23.82854930 |
Nekro Virus | Advanced | 23.31099933 |
Clan of Saar | Intermediate | 23.21175442 |
Ghosts of Creuss | Advanced | 22.79161677 |
Arborec | Advanced | 22.70521399 |
L1Z1X Mindnet | Intermediate | 22.49837161 |
Winnu | Advanced | 22.34945804 |
Xxcha Kingdom | Intermediate | 22.01560255 |
Yssaril Tribes | Intermediate | 21.98374204 |
Naalu Collective | Intermediate | 21.97271666 |
Yin Brotherhood | Intermediate | 21.74583770 |
Sardakk N’orr | Advanced | 21.58468850 |
Ghosts of Creuss | Intermediate | 20.38895273 |
Mentak Coalition | Intermediate | 20.11534065 |
Nekro Virus | Intermediate | 19.92369537 |
Embers of Muaat | Intermediate | 19.51033925 |
Arborec | Intermediate | 19.49624435 |
Winnu | Intermediate | 18.58748763 |
Federation of Sol | Beginner | 18.06861607 |
Universities of Jol-Nar | Beginner | 17.94425337 |
Sardakk N’orr | Intermediate | 17.67737576 |
Emirates of Hacan | Beginner | 17.26997944 |
Naalu Collective | Beginner | 15.92627194 |
L1Z1X Mindnet | Beginner | 14.49165522 |
Clan of Saar | Beginner | 13.89991205 |
Mentak Coalition | Beginner | 13.84376820 |
Yssaril Tribes | Beginner | 13.54292539 |
Barony of Letnev | Beginner | 13.43430308 |
Winnu | Beginner | 12.64705960 |
Xxcha Kingdom | Beginner | 12.26642650 |
Yin Brotherhood | Beginner | 11.40798903 |
Nekro Virus | Beginner | 10.97676076 |
Embers of Muaat | Beginner | 10.93323739 |
Ghosts of Creuss | Beginner | 10.32771177 |
Arborec | Beginner | 10.21294854 |
Sardakk N’orr | Beginner | 9.38140506 |
Race | Skill | Rating |
---|---|---|
Federation of Sol | Advanced | 28.35842920 |
Clan of Saar | Advanced | 22.19586105 |
Naalu Collective | Advanced | 22.07393017 |
Universities of Jol-Nar | Intermediate | 21.58011268 |
Barony of Letnev | Intermediate | 20.04751044 |
Federation of Sol | Intermediate | 19.79323260 |
Clan of Saar | Intermediate | 19.75206821 |
Naalu Collective | Intermediate | 19.49237002 |
Yin Brotherhood | Intermediate | 19.12075107 |
Yin Brotherhood | Advanced | 18.24795462 |
Universities of Jol-Nar | Advanced | 17.67682680 |
Winnu | Advanced | 17.59674256 |
Sardakk N’orr | Intermediate | 17.09480067 |
Barony of Letnev | Advanced | 16.94918973 |
Yssaril Tribes | Intermediate | 16.82426022 |
Emirates of Hacan | Advanced | 16.71328359 |
Xxcha Kingdom | Intermediate | 16.65088203 |
Arborec | Advanced | 15.82658929 |
L1Z1X Mindnet | Advanced | 15.28411227 |
Universities of Jol-Nar | Beginner | 15.15283050 |
Arborec | Intermediate | 14.83767142 |
Ghosts of Creuss | Intermediate | 14.83303881 |
Mentak Coalition | Advanced | 14.46162133 |
Emirates of Hacan | Intermediate | 14.29516927 |
L1Z1X Mindnet | Intermediate | 14.25209345 |
Naalu Collective | Beginner | 14.23268881 |
Winnu | Intermediate | 13.29475026 |
Embers of Muaat | Intermediate | 13.21816327 |
Federation of Sol | Beginner | 12.97932160 |
Mentak Coalition | Intermediate | 12.94256326 |
Xxcha Kingdom | Advanced | 12.77991692 |
Sardakk N’orr | Advanced | 12.34996131 |
Yssaril Tribes | Beginner | 12.02002843 |
Barony of Letnev | Beginner | 11.98649407 |
Clan of Saar | Beginner | 11.72262389 |
Emirates of Hacan | Beginner | 11.35077961 |
Embers of Muaat | Advanced | 10.85411247 |
Yssaril Tribes | Advanced | 10.46650665 |
Winnu | Beginner | 10.24769693 |
Arborec | Beginner | 10.13040202 |
Nekro Virus | Beginner | 8.99587795 |
L1Z1X Mindnet | Beginner | 8.88800772 |
Ghosts of Creuss | Beginner | 8.85522639 |
Nekro Virus | Intermediate | 8.54002581 |
Mentak Coalition | Beginner | 8.53487268 |
Xxcha Kingdom | Beginner | 7.79029366 |
Ghosts of Creuss | Advanced | 6.52756990 |
Nekro Virus | Advanced | 5.99907088 |
Sardakk N’orr | Beginner | 5.10565191 |
Embers of Muaat | Beginner | 4.72155823 |
Yin Brotherhood | Beginner | 1.62033691 |
Race | Skill | Rating |
---|---|---|
Emirates of Hacan | Advanced | 27.41685857 |
Federation of Sol | Advanced | 27.12632515 |
Universities of Jol-Nar | Advanced | 25.98764682 |
Barony of Letnev | Advanced | 24.75410318 |
Yssaril Tribes | Advanced | 24.65746627 |
Clan of Saar | Intermediate | 24.41965864 |
Clan of Saar | Advanced | 24.36933895 |
Ghosts of Creuss | Advanced | 23.67478226 |
Naalu Collective | Intermediate | 23.06349920 |
Arborec | Advanced | 22.34279091 |
Federation of Sol | Intermediate | 22.11128085 |
Barony of Letnev | Intermediate | 21.55657956 |
Universities of Jol-Nar | Intermediate | 21.14545395 |
Emirates of Hacan | Intermediate | 20.86726212 |
L1Z1X Mindnet | Intermediate | 20.78223587 |
Naalu Collective | Advanced | 20.71476420 |
L1Z1X Mindnet | Advanced | 20.30099494 |
Nekro Virus | Intermediate | 20.04592528 |
Nekro Virus | Advanced | 19.14202315 |
Xxcha Kingdom | Advanced | 18.98693015 |
Ghosts of Creuss | Intermediate | 18.69450804 |
Mentak Coalition | Advanced | 18.68118344 |
Xxcha Kingdom | Intermediate | 18.56008902 |
Yssaril Tribes | Intermediate | 18.48626891 |
Mentak Coalition | Intermediate | 17.84763184 |
Winnu | Advanced | 17.28844185 |
Arborec | Intermediate | 16.69693677 |
Embers of Muaat | Advanced | 16.66367115 |
Yin Brotherhood | Advanced | 16.59169254 |
Yin Brotherhood | Intermediate | 15.83040628 |
Universities of Jol-Nar | Beginner | 15.81007492 |
Embers of Muaat | Intermediate | 15.47055078 |
Federation of Sol | Beginner | 14.67140062 |
Emirates of Hacan | Beginner | 14.62092735 |
Sardakk N’orr | Advanced | 14.10729327 |
Winnu | Intermediate | 13.12065630 |
Sardakk N’orr | Intermediate | 12.24657652 |
L1Z1X Mindnet | Beginner | 12.15983136 |
Barony of Letnev | Beginner | 11.80266354 |
Yssaril Tribes | Beginner | 11.16116908 |
Xxcha Kingdom | Beginner | 11.10725323 |
Naalu Collective | Beginner | 10.18696673 |
Mentak Coalition | Beginner | 9.99142409 |
Ghosts of Creuss | Beginner | 9.13803070 |
Winnu | Beginner | 9.09935158 |
Clan of Saar | Beginner | 8.93580276 |
Arborec | Beginner | 7.08075890 |
Yin Brotherhood | Beginner | 6.71407429 |
Sardakk N’orr | Beginner | 6.66415085 |
Embers of Muaat | Beginner | 6.58220814 |
Nekro Virus | Beginner | 5.94971620 |
Race | Skill | Rating |
---|---|---|
Universities of Jol-Nar | Advanced | 26.40951856 |
Naalu Collective | Advanced | 26.09599426 |
Federation of Sol | Advanced | 24.28649281 |
Yssaril Tribes | Advanced | 23.31078670 |
Universities of Jol-Nar | Intermediate | 23.03156183 |
Mentak Coalition | Advanced | 22.88911195 |
Xxcha Kingdom | Advanced | 22.70454211 |
L1Z1X Mindnet | Advanced | 22.47887765 |
Emirates of Hacan | Advanced | 22.46079117 |
Ghosts of Creuss | Advanced | 22.36559566 |
Federation of Sol | Intermediate | 20.84047316 |
Clan of Saar | Advanced | 20.09023957 |
L1Z1X Mindnet | Intermediate | 20.02042594 |
Embers of Muaat | Advanced | 19.73944017 |
Emirates of Hacan | Intermediate | 19.55730314 |
Yssaril Tribes | Intermediate | 18.92498108 |
Yin Brotherhood | Intermediate | 18.76434052 |
Xxcha Kingdom | Intermediate | 18.69266094 |
Yin Brotherhood | Advanced | 17.82495338 |
Arborec | Advanced | 17.74965882 |
Naalu Collective | Intermediate | 17.15486551 |
Sardakk N’orr | Advanced | 17.11566967 |
Clan of Saar | Intermediate | 16.64206843 |
Barony of Letnev | Intermediate | 16.33207741 |
Nekro Virus | Advanced | 16.19982226 |
Arborec | Intermediate | 15.55468130 |
Mentak Coalition | Intermediate | 15.42719412 |
Federation of Sol | Beginner | 15.34569621 |
Barony of Letnev | Advanced | 15.33413489 |
Embers of Muaat | Intermediate | 15.32630181 |
Ghosts of Creuss | Intermediate | 15.11307261 |
Nekro Virus | Intermediate | 14.87724946 |
Winnu | Advanced | 12.82994599 |
Emirates of Hacan | Beginner | 12.76963523 |
Universities of Jol-Nar | Beginner | 12.63083834 |
Winnu | Intermediate | 11.70944388 |
Naalu Collective | Beginner | 11.50965206 |
L1Z1X Mindnet | Beginner | 10.19190402 |
Mentak Coalition | Beginner | 10.15234779 |
Sardakk N’orr | Intermediate | 10.09528497 |
Yssaril Tribes | Beginner | 9.011886311 |
Clan of Saar | Beginner | 8.642295667 |
Yin Brotherhood | Beginner | 8.174616094 |
Xxcha Kingdom | Beginner | 7.595298702 |
Nekro Virus | Beginner | 7.269565769 |
Embers of Muaat | Beginner | 6.636267959 |
Barony of Letnev | Beginner | 5.982703736 |
Ghosts of Creuss | Beginner | 4.545880900 |
Sardakk N’orr | Beginner | 2.233827176 |
Winnu | Beginner | 1.861867246 |
Arborec | Beginner | -0.313055713 |
Race | Skill | Rating |
---|---|---|
Universities of Jol-Nar | Advanced | 29.30604802 |
Federation of Sol | Advanced | 28.79152602 |
Yssaril Tribes | Advanced | 28.26945873 |
Emirates of Hacan | Advanced | 26.78807763 |
Naalu Collective | Advanced | 26.29754493 |
L1Z1X Mindnet | Advanced | 26.27279148 |
Xxcha Kingdom | Advanced | 24.65932500 |
Mentak Coalition | Advanced | 24.51731564 |
Emirates of Hacan | Intermediate | 24.43834184 |
Universities of Jol-Nar | Intermediate | 23.88263850 |
Federation of Sol | Intermediate | 23.50092211 |
Clan of Saar | Advanced | 22.64128027 |
Embers of Muaat | Advanced | 22.29999029 |
Barony of Letnev | Intermediate | 21.88630842 |
Nekro Virus | Advanced | 21.36942925 |
Clan of Saar | Intermediate | 20.66955656 |
Yin Brotherhood | Advanced | 20.42376125 |
Barony of Letnev | Advanced | 20.09158084 |
Yssaril Tribes | Intermediate | 19.70508387 |
L1Z1X Mindnet | Intermediate | 19.15440624 |
Mentak Coalition | Intermediate | 18.76436872 |
Winnu | Advanced | 18.71805711 |
Sardakk N’orr | Advanced | 18.27876607 |
Naalu Collective | Intermediate | 17.83173565 |
Xxcha Kingdom | Intermediate | 17.71077249 |
Ghosts of Creuss | Advanced | 16.95129573 |
Yin Brotherhood | Intermediate | 16.34530525 |
Federation of Sol | Beginner | 16.30904960 |
Arborec | Advanced | 15.88364406 |
Ghosts of Creuss | Intermediate | 15.85997772 |
Winnu | Intermediate | 15.73101650 |
Embers of Muaat | Intermediate | 15.63534945 |
Nekro Virus | Intermediate | 15.44865376 |
Arborec | Intermediate | 14.97098940 |
Emirates of Hacan | Beginner | 14.66100957 |
Universities of Jol-Nar | Beginner | 14.22018825 |
Sardakk N’orr | Intermediate | 13.84072241 |
Naalu Collective | Beginner | 13.69977576 |
Clan of Saar | Beginner | 9.07924681 |
L1Z1X Mindnet | Beginner | 7.88780919 |
Barony of Letnev | Beginner | 7.58641659 |
Mentak Coalition | Beginner | 6.71237862 |
Xxcha Kingdom | Beginner | 5.06243542 |
Arborec | Beginner | 4.77849320 |
Embers of Muaat | Beginner | 3.59396214 |
Yin Brotherhood | Beginner | 3.58529868 |
Nekro Virus | Beginner | 3.17071658 |
Yssaril Tribes | Beginner | 3.06349557 |
Winnu | Beginner | 2.87900750 |
Sardakk N’orr | Beginner | 2.21474336 |
Ghosts of Creuss | Beginner | 0.85199654 |
If you consider yourself advanced and draw Letnev, don’t try to be fancy. Just get capital ships.
While there is a pronounced gradient in skill across different races, player experience is an order of magnitude more important for predicting a winner. That said, experience isn’t something you can easily control and race selection is effectively an opening move. An unpopular opening like selecting Winnu isn’t necessarily bad if your opponents are unfamiliar with it. Selecting them in a 3 player game might throw your opponents on tilt and score a win.
This shouldn’t invalidate the golden rule: Play to have fun. In a friendly game, pick a race you think you’ll have fun playing. If you really want a Death StarWar Sun, go with Muaat and if you win, know that you did it against the odds. Imperically this data should show the races are more balanced than it may appear, which is a good thing for a game that most players may only get to play a handful of times a year.
ssh-keygen
will generate 3072-bit RSA keys, equivalent to a 128-bit symmetric key, which people smarter than me say is sufficient.
One of these keys might look like this:
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCo1/2nxzBea7BkBJmbPUO3fW7HYiUIS+85PuycJ36z
iz/TP44IBkCFyFZxBanGDWfSJDL/L2cWJ21c3S9iRvDx2skKG7ZUHO04bWXeFcYoVhUKNrusj2bipy3N
Q+uIDKLalYhoZvQA1qdrnO91V4GIcmXb94NTaOofT3TUVXeFyOuMHygMM86eUUppFy/j6B6lpIx52S3L
utB4xV3Istgi+9hogkwRpcEOcQWXkwQIMWk1hJrNVw3u17Y0CLv1c1AOGhz1bdpRrnc20nOH3OWj6VW1
Qcd9VgVQx+JmtwBB4sUpm79shlv8tt0K/JxQ19zc9R/DHD2MXCPuHP9KDZ8+hpJWzyGAo1Q6/nPhHt6g
zWAPsOw20EZ0XO9l2onCknnfcOQJdyYoW5Klqa2ZgGpfuWo2yK2cXaAv2rZgrw56CCeQbbLgNLT1Jsbn
Q7VsILJBCg/zr+VoEbYFiGhixWWdTzqurcaozsoRcYzlMUV6SZOi3dCvTeLcOhyXKnieYJU=
As a binary embedded in a QR code, this could look like this:
If sufficient is good enough for you, then there is no reason to read this post, but if you want to be a special snowflake, then upgrade to an ed25519 key!
Ed25519 keys are really short because instead of your key being two prime factors, you can just use any random noise as a private key, and generate your corresponding public key from that. Generate them like this:
ssh-keygen -t ed25519
This creates a really slim key, so in all of your authorized_keys
files, you’ll just look like
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIO/hBOfgryiHaeNkhjwehgKWIyTgNAvHbPiNPCrCyWd5
One line! So compact! Turn this into a QR code and it looks like
There are absolutely valid reasons for you to do this for security reasons, but none of these reasons is urgent. The thing that’s urgent for right now is vanity. Who wants a large 4096-bit key when you can have a key 10% that size, and perhaps more secure? I’ll tell you who. You. You want that.
These are also compatible with GnuPG
]]>Configuring Mutt out of the box requires you to put your plaintext password in a ~/.muttrc
file, which is asking for all sorts of problems. The following is how I use a hardware Yubikey to protect my Gmail password, and then sync that password across machines.
If you don’t already have one, get a Yubikey. Here are some options:
While waiting for this to arrive, you can continue with a GPG key on your hard drive; just make sure you protect it with a passphrase, which GPG will strongly encourage, because otherwise there’s no point.
If you’ve created your key already, migrate it to the card. Most people just have one master key, and one encryption subkey; if you run gpg --list-secret-keys
, you may see something like this:
/Users/philihp/.gnupg/pubring.kbx
---------------------------------
sec rsa4096/0x5B640B9F9600F122 2016-02-29 [SC] [expires: 2021-02-07]
427E032939DB40F29D03D80F5B640B9F9600F122
uid [ultimate] Philihp Busby <philihp@gmail.com>
ssb rsa4096/0x0D86EF2BF0DA842E 2016-02-29 [E] [expires: 2021-02-07]
sec
means “i have the secret key”ssb
means “i have the secret subkey”.pub
or sub
, it means “i just have the public key”, and that’s a problem.[E]
subkeys.In the brackets in the 4th column, you can see [SC]
for the master key meaning it is meant for the “Signing” usage and the “Certification” usage, and [E]
for the subkey meaning it is meant for “Encryption”. I think it’s not a bad idea to create another subkey for “Authentication” or add that usage to an existing key, but important: , but there are two important points:
S
signing keys or A
authentication keys.E
usage. When a message is encrypted, GPG uses the newest E
subkey, i.e. the one with the last creation.To move these to your Yubikey, run the command
gpg --edit-key 5B640B9F9600F122
where 5B640B9F9600F122
is your key… it’s actually my key; I don’t know your key. You could tell me it, though, I’d love to know if this helped you. So if you run that, you’ll be dropped into another console
gpg (GnuPG) 2.2.20; Copyright (C) 2020 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Secret key is available.
...
...
gpg>
Useful commands here: help
, for common commands; list
to show your key, key N
, to select a subkey where N is the index number of the key starting with 1, and keytocard
to move the selected key to the card.
!> If you don’t have a key selected, keytocard
will move the master key.
keytocard
without a key selected to move your master key into the Signing slot of your Yubikey.gpg> keytocard
Really move the primary key? (y/N) y
Please select where to store the key:
(1) Signature key
(3) Authentication key
Your selection? 1
Replace existing key? (y/N) y
key 1
and keytocard
to move your encryption subkey into the Encryption slot of your Yubikeygpg> key 1
...
sec rsa4096/0x5B640B9F9600F122
...
ssb* rsa4096/0x0D86EF2BF0DA842E
...
ssb rsa4096/0xFD8194C54A63DBD5
...
gpg> keytocard
Please select where to store the key:
(2) Encryption key
Your selection? 2
change-usage
, move this into the Authentication slot of your Yubikey.save
to save your key.Now running
gpg --list-secret-keys
Should display something like
sec> rsa4096/0x5B640B9F9600F122 2016-02-29 [SC] [expires: 2021-02-07]
427E032939DB40F29D03D80F5B640B9F9600F122
Card serial no. = 0006 09123456
uid [ultimate] Philihp Busby <philihp@gmail.com>
ssb> rsa4096/0x0D86EF2BF0DA842E 2016-02-29 [E] [expires: 2021-02-07]
ssb> rsa4096/0xFD8194C54A63DBD5 2020-06-13 [A] [expires: 2021-02-07]
Some things that changed here:
sec
turned into sec>
, meaning “I know where to get the private key”ssb
turned into ssb>
, similarly.Card serial no. = 0006 09123456
meaning “It’s on a smart card
with this serial number.gpg-agent
as a replacement for ssh-agent
)Similarly, if you run
gpg --card-status
You should see something like
❯ gpg --card-status
Reader ...........: Yubico YubiKey FIDO CCID
Application ID ...: D2760001240102010006091234560000
Application type .: OpenPGP
Version ..........: 2.1
Manufacturer .....: Yubico
Serial number ....: 09123456
Name of cardholder: Philihp Busby
Language prefs ...: en
Salutation .......:
URL of public key : https://philihp.com/pgp.asc
Login data .......: philihp
Signature PIN ....: forced
Key attributes ...: rsa4096 rsa4096 rsa4096
Max. PIN lengths .: 127 127 127
PIN retry counter : 3 0 3
Signature counter : 42
Signature key ....: 427E 0329 39DB 40F2 9D03 D80F 5B64 0B9F 9600 F122
created ....: 2016-02-29 07:34:57
Encryption key....: C54A 7C6F 8B38 A89F 3102 4BAB 0D86 EF2B F0DA 842E
created ....: 2016-02-29 07:34:57
Authentication key: D5CB FB11 287E 0B3A 287D F591 FD81 94C5 4A63 DBD5
created ....: 2020-06-13 04:53:06
General key info..: pub rsa4096/0x5B640B9F9600F122 2016-02-29 Philihp Busby <philihp@gmail.com>
sec> rsa4096/0x5B640B9F9600F122 created: 2016-02-29 expires: 2021-06-12
card-no: 0006 09123456
ssb> rsa4096/0x0D86EF2BF0DA842E created: 2016-02-29 expires: 2021-02-07
card-no: 0006 09123456
ssb> rsa4096/0xFD8194C54A63DBD5 created: 2016-11-08 expires: 2021-02-07
card-no: 0006 09123456
If you don’t have a GPG key, or you don’t want to reuse it, this is simpler.
gpg --edit-card
Should bring you into a console for tinkering with the card. The help
command will show you most of the common commands anyone should need. Most of the fun stuff is enabled with the admin
command, which you should do.
gpg/card> admin
Admin commands are allowed
From there, you might want to use the factory-reset
command, or at least know it’s there. passwd
will let you change the PIN (default: 123456), or the Admin PIN (default: 12345678).
Run the following two commands:
gpg/card> admin
Admin commands are allowed
gpg/card> generate
It will ask you if you want an off-key backup, which it’s not a bad idea to do, and take the file in ~/.gnupg/sk_????????????????.gpg
and the similar file in ~/.gnupg/openpgp-revocs.d/*
, and copy those to a reliable and secure location. Treat these like you would treat your passport. You can get another, but you really don’t want to because you will lose all of your visas and history.
You should now be able to sign something like a git commit… Configure it like this (with your own key, of course)
git config --global commit.gpgsign true
git config --global user.signingkey 427E032939DB40F29D03D80F5B640B9F9600F122
git config --global log.showSignature true
Then go commit somewhere; it should ask you for a pin. Then git log
and you can see your signature.
commit a7558903018908258386c9cdabca70e47c6aed24 (HEAD -> master, origin/master)
gpg: Signature made Fri Jun 12 03:26:18 2020 GMT
gpg: using RSA key D5CBFB11287E0B3A287DF591FD8194C54A63DBD5
gpg: Good signature from "Philihp Busby <philihp@gmail.com>" [ultimate]
Author: Philihp Busby <philihp@gmail.com>
Date: Fri Jun 12 03:26:18 2020 +0000
Committing like a pro!
The UI you get for PIN entry is a ncurses-driven text-based PIN entry. This can cause issues with other command line programs, or weird behavior if you commit with a GUI.
I like to use a GUI-driven PIN entry as another layer of being sure I’m not being spoofed into unlocking my Yubikey; setting that up is as simple as
brew install pinentry-mac
echo "pinentry-program /usr/local/bin/pinentry-mac" >> ~/.gnupg/gpg-agent.conf
and then kill any existing gpg-agent process.
Now that that’s sorted, it’s time to setup Passwordstore! Passwordstore is a 719 line shell script that fulfills a lot of the same functions as any commercial password manager. It’s a posterchild for the Unix philosophy by delegating encryption bits to GPG, and delegating the syncing of filesystems with nonlinear history to git. When decrypting a password, it gives it back to you in STDOUT, so you can pipe it as well. And it’s short, so if that’s not enough for you, or you’re wondering how it works (as I once did when I wanted to use it in a team setting, which it does very well), you can read it and get a pretty good grasp on it. I love it. Setting it up is pretty easy. On macOS, just run
brew install pass
and then
pass init 427E032939DB40F29D03D80F5B640B9F9600F122
This will create a folder ~/.password-store
, with a file .gpg-id
of with your ID in it. If you want other people (perhaps people on your team) to be able to decrypt the files here too, add them as a new line.
Now go to your Google Account and create an App Password. You should be given a 16 character lowercase password. Tell that to Pass with
pass insert gmail.com/philihp@gmail.com
Then if you go to ~/.password-store/gmail.com/
you should see a file philihp@gmail.com.gpg
. All this is is a text file with your password, encrypted. Test out decrypting it with:
gpg --decrypt ~/.password-store/gmail.com/philihp@gmail.com.gpg
Another way to get that is with
pass gmail.com/philihp@gmail.com
or if you were going to paste this somewhere and didn’t want it displayed to the screen,
pass -c gmail.com/philihp@gmail.com
Cool. Now you can programmatically request your own password from the command line. Now if we had a highly configurable email client, we could tell it a command that goes and gets your password so it can login and show you your email.
If you want to sync this with another machine, you can run
pass git init
pass git remote add origin philihp@dolores.local:password-store.git
pass git push
And on your other machines, instead of reconfiguring this, you can just say
git clone philihp@dolores.local:password-store.git ~/.password-store
Install mutt (or neomutt) with
brew install mutt
Then configure it by creating a file ~/.muttrc
with
set realname = "Philihp Busby"
set from = "philihp@gmail.com"
set use_from = yes
set envelope_from = yes
set smtp_url = "smtps://philihp@gmail.com@smtp.gmail.com:465/"
set smtp_pass = `pass show gmail.com/philihp@gmail.com`
set imap_user = "philihp@gmail.com"
set imap_pass = `pass show gmail.com/philihp@gmail.com`
set folder = "imaps://imap.gmail.com:993"
set spoolfile = "+INBOX"
set ssl_force_tls = yes
!> Notice the back-ticks on the smtp_pass
and imap_pass
. That’s important, because it tells mutt that rather than “this is the password”, it says to execute that and it will give you back the password.
Now startup mutt with
mutt
If doesn’t manage to get your gmail password with this command, it will ask you for it.
I hope I didn’t expose myself by writing this, and admittedly this is a very unique snowflake for email; most people just use a browser on gmail.com. This setup has evolved over a couple years, and works very well for me.
There are some more things you can configure, which are not simple with other setups:
Automatically sign and verify signatures, or encrypt and decrypt emails (but perhaps don’t always sign)
Replace ssh-agent
with gpg-agent
and require your Yubikey to SSH or SCP.
Configure your Gmail to use your Yubikey to login to the web
Tell Github about your PGP key, so you’ll get a fancypants “Verified” badge next to your commits.