Lazy evaluation in python: zipping infinite lists

One demonstration of lazy evaluation in a programming language is to create an infinitely long list containing the Fibonacci sequence, defined recursively as the concatenation of two lists:

  • the list [1, 1]
  • a list made by zipping the Fibonacci sequence with the tail of the Fibonacci sequence, and summing the resulting pairs

The canonical example code in Haskell is:

What is memcache?

memcached is a small executable program. In the most idiomatic deployment, it listens for TCP connections on port 11211. Clients that connect have access to read and write a single in-memory hashtable with short (up to 250 byte) binary keys and longer (up to 1 megabyte) binary values. Clients use a simple text based protocol to store values in the hashtable by key, or get them by key.

memcached makes no guarantees about how long it will store a key/value pair. If you store a key/value, it may or not be there later when you attempt to retrieve it.

Composable asset and vulnerability expected value model for security

Security is often thought of as a level of risk to "assets". An asset is anything of value, like a password, 1000 hours of CPU time, the ability to execute software on a machine, customer credit card numbers, employee Social Security Numbers, etc.

A "vulnerability" can be thought of as weaknesses that an attacker who already has access to assets X1, X2, X3, ... can exploit to gain access to new assets Y1, Y2.

Exporting a graph from neo4j and importing it into gephi

Some code to calculate the <key> elements needed to get Gephi to import the labels for nodes and edges exported by neo4j

Run neo4j-shell inside docker

To start the neo4j docker image with neo4j-shell enabled, run:

docker run \
--publish=7474:7474 --publish=7687:7687 \
--volume=$HOME/neo4j/data:/data \
neo4j sh -c "cd /var/lib/neo4j; sed -i -e 's/' conf/neo4j.conf; curl -o; unzip -o -d lib; / neo4j" 

Rebase all local branches in git

To rebase all local branches on upstream/master in git:

git for-each-ref --shell --format="git rebase $BASEBRANCH %(refname:short)" refs/heads/ | sh -x

Query a server certificate from the command line

To display SSL/TLS certificate information about a web server from the command line:

  : | openssl s_client -showcerts -servername "$HOSTNAME" -connect "$HOSTNAME":443 2>/dev/null | openssl x509 -inform pem -noout -text

Then, call this function with the hostname you want to check, like:


Apache Beam Coder classes

(Last updated: API version 2.0.0)

Coder classes in Beam encapsulate the functionality of serializing and deserializing the values that you might find in a PCollection. Every PCollection has a Coder class associated with it, either because it was set explicitly, or inferred from the type of the values in the PCollection.

To get the default Coder for a given type, you can call:

Apache Beam State API

(Last updated: API version 2.0.0)

Beam exposes a State API that lets DoFn classes read and write arbitrary state that persists across individual stream elements. The DoFn classes need to be of type DoFn<KV<Key, Value>, ?> and a separate state will be maintained for each distinct value of Key. For example here is a DoFn that maintains a separate counter for each distinct integer key in its input PCollection:

Kappa Architecture

The actor model, functional programming (including both Map Reduce flavor and Haskell streaming flavor) and materialized views seem to be merging together and producing love children that the cool kids are calling reactive programming and the "kappa" architecture.

This talk about Apache Samza had an insight was new to me -- that you could conceptually think of your video memory in your graphics card as


Subscribe to Wesley Tanaka RSS