wesley tanaka

Apache Beam Coder classes

(Last updated: API version 0.6.0)

Coder classes in Beam encapsulate the functionality of serializing and deserializing the values that you might find in a PCollection. Every PCollection has a Coder class associated with it, either because it was set explicitly, or inferred from the type of the values in the PCollection.

To get the default Coder for a given type, you can call:

Apache Beam State API

Beam exposes a State API that lets DoFn classes read and write arbitrary state that persists across individual stream elements. The DoFn classes need to be of type DoFn<KV<Key, Value>, ?> and a separate state will be maintained for each distinct value of Key. For example here is a DoFn that maintains a separate counter for each distinct integer key in its input PCollection:

Kappa Architecture

()

The actor model, functional programming (including both Map Reduce flavor and Haskell streaming flavor) and materialized views seem to be merging together and producing love children that the cool kids are calling reactive programming and the "kappa" architecture.

This talk about Apache Samza had an insight was new to me -- that you could conceptually think of your video memory in your graphics card as

UnsupportedJavaRuntimeException.assertUsingVersion in Android Studio

()

After upgrading Android Studio to version 2.3, you may get:

Experiment

Empirical evidence that our baking powder is no good.

Against Empathy

()

Paul Bloom gave an interview on EconTalk to promote his book Against Empathy: The Case for Rational Compassion and talk about why empathy (as opposed to compassion) is a leading motivator of inequality and immorality in society.

The quote about empathy in parenting decision making particularly struck me:

Decoding binary data in logstash

()

By default, logstash stores the "message" field as a unicode string (in this example, we avoid including a '\x0a' byte to keep the pipeline configuration shorter):

Downloading Ansible galaxy roles in parallel

()

To always ensure that I have the latest version of a role from Ansible Galaxy, I've found I needed to use both the --force option (to re-download even if the directory already exists) and the --ignore-errors option (to keep going through all the roles). Overall, the command line ends up looking like this:

ansible-galaxy install --force --ignore-errors -r priv/ansible/requirements.txt -p priv/ansible/roles/

letsencrypt / certbot incomplete certificate chain error

()

Some tutorials on enabling HTTPS in Apache recommend adding this in the <VirtualHost *:443> configuration block:

SSLEngine on
SSLCertificateFile "/etc/letsencrypt/live/wtanaka.com/fullchain.pem"
SSLCertificateKeyFile "/etc/letsencrypt/live/wtanaka.com/privkey.pem"

When this configuration is used with Apache versions prior to 2.4.8, Apache is not able to parse the intermediate CA out of a chain certificate.

Overview of Kano OS

()

We bought a computer for our son. Tonight I installed Kano OS 3.8.0 and experimented to evaluate whether to use that as his operating system.  My impressions:

Syndicate content
by Wesley Tanaka