Publishing to Elasticsearch using Django and Postgres DictCursor

One of the projects I've been working on relies heavily on elasticsearch and the current dataset is a few million records. I'm currently indexing about a million records and expect to index the entire data set in the near future. I originally took a simple approach to indexing the data which took about an hour and a half to fully index. The data orginates from an external source and I&…

Term-centric search using Elasticsearch and Django

Recently I've been working on a side project that has a search component at it's core. I'm pulling a set of data from an external source, loading it into my database and pushing the data to Elasticsearch. There are a handful of fields that I need to search over and I wanted to do a google style search (single search bar) rather than a faceted search. This means I have to search for…

Setting Up Basic Authentication with Elasticsearch

I've been working with elasticsearch a lot lately and have been really enjoying it. It has an awesome api and so many great built in tools. They make building complex search logic really enjoyable and the documentation is amazing. One of the downsides of using elasticsearch is they don't have any built in authentication. One way of handling this is to setup firewall rules or ufw rules …

Running Elasticsearch in a Docker Container

The current project I'm working on has a heavy dependency on Elasticsearch and I thought it would be good to walk through my setup using Docker. I wanted to manage Elasticsearch using Docker for a few reasons:

  1. I usually have multiple projects running on my laptop. I love the idea of being able to manage services on a per pro…