Having gained some insights into OpenTelemetry through previous experiments, we planned to forward 50x nginx log records to an aggregator through OpenTelemetry. read more...

As Web traffic has spiked other the summer, the engineering team is looking to upgrade the logging infrastructure to have better visibility into "soft errors" - i.e. 502, 503, 504 and the like. The team has been more and more relying on a self-hosted GitLab instance for day-to-day activity, and has seen many new features around Monitoring coming out of GitLab. Soon the idea of shipping OpenTelemetry events to GitLab started to look good. read more...

Over time we ran production workloads on Fedora, CentOS7, Amazon Linux 2, and now Amazon Linux 2023 (which means we are back on Fedora I guess). This post is about the quirks we encountered migrating from Amazon Linux 2 to Amazon Linux 2023. read more...

After experimenting with AI-generated voice over in a previous post, it was time to amp it up. In this post, we will sign up for Synthesia and attempt to generate a picture-in-picture of a AI-generated actor taking us through a video tutorial. read more...

Many times we stopped an EC2 instance, changed the instance type and started it again. This time, the instance would just refuse to start again. Let's dive in how we debugged this issue and fixed it. read more...

A latest update of Jenkins (version 3.6) triggered an required upgrade to Java 11. The process to upgrade from Java8 to Java11 on Amazon Linux2 is pretty straightforward. read more...

You have decided to launch your SaaS product and are looking forward to subscribers signing up with their credit card. Congratulations! Let's now see the transactional e-mails your customers will be expecting. read more...

I have a prod test-bed with privileged ports, FQDNs, and production configs running in my home lab. With it, I can test all our apps, proxies, databases, and unattended install scripts without having to deploy a new instance. In this article, I'll explain how I set up an Amazon Linux 2 virtual machine on my headless home server, and how I used WireGuard to elevate that VM from a handy sandbox to a first-class, low-maintenance development tool. read more...

The main reason to implement a fast path to upload logs is to be able to process them quicker than through the daily batch process. Fortunately AWS S3 can generate events into a SQS queue and/or trigger Lambda function when a new file is created in an S3 bucket. read more...

Many times debugging logrotate scripts rely on making changes and waiting for the next day to check the results. Today we will see how to be more efficient and debug logrotate scripts until they work. read more...

Before you start any DevOps workflow, you will need to pick an OS distribution (and image) to base the servers on. read more...

Through the years and Jenkins releases, we build Jenkins jobs that are able to run tests on the master branch. With more and more casual contributors, we want to be able to run tests on Pull Requests, automatically adding annotations for the test results. read more...

Django 2.2 was released on Apr 1st 2019. Since it is the most recent as well a Long Term Support (LTS) version, it was time to update the Jenkins bot infrastructure to include Django 2.2 in the Django versions tested. read more...

With growth comes many support issues, one of them being able to efficiently answer questions for developers and prospective customers. The API doc hosted on Read-the-Docs were showing their limitations. So we embark onto surveying the landscape for API documentation, looking for a solution that fits DjaoApp APIs implemented with Django Rest Framework. read more...

Sometime ago, I wrote about installing postgresql on an encrypted volume and running postgresql on an ebs volume encrypted with keys managed by AWS KMS. Today let's see how we can quickly deploy postgresql on an RDS instance with both encryption at-rest and in-transit. read more...

The major features released in Jenkins v2 are Jenkinsfile and Pipelines. Today, we will see how to create a Jenkins job to build a docker container out of a source repository then upload it to an Amazon EC2 Container Registry. read more...

It has been a while since we setup Nginx, Jetty and Jenkins together the first time around. Fedora is on version 25, Jetty on version 9 and Jenkins on version 2.60. read more...

Charge receipts are an essential part of payments online, both for selling physical goods (e-commnerce) and subscriptions for Software-as-a-Service (SaaS). A good receipt will help customers build trust in your business as well as avoid chargebacks. In this post, let's review the elements needed for a good receipt. read more...

Sometimes you underestimated the disk size required to setup and run a service on AWS. The quickest way to keep moving forward is to resize the EBS storage associated with the EC2 instance. read more...

As we are getting ready for a Django 1.10 upgrade, it is time to take care of one of the most baffling warning in the code base RemovedInDjango110Warning: SubfieldBase has been deprecated. This lead to a long journey through code, documentation, and release notes. read more...

While deploying code on a production server can be tricky due to prerequisites, version drift, etc. Serving static assets, commonly called Plain Old Data (POD) has its own set of challenges. Often static assets are "compiled" (gzip, minification), moved around the filesystem and served to a browser by completely different services (ex: nginx) than during development. In practice it means a lot of things can potentially go wrong. read more...

Dealing with large files over HTTP has always been challenging, Doing so in the context of access control and user authentication even more so. Once faced with that first (too) big file, you will most likely search for Nginx direct file upload without passing them through backend or if you rely on AWS, look into S3 direct upload from browser. We will see in this post how to do the latest. read more...

Time for an upgrade of the Jenkins machine. This time we are setting up Jenkins with Jetty 9 on Fedora 21 to continuously run CasperJS tests against the Django web application. read more...

Amazon announced new T2 instances in July 2014. A little later it became clear that AWS free tier for StartUps was only for T2 instances. The free tier offer did not extend to the previous T1 generation (We found out on the first bill). read more...

In this post I will describe how we organize DjaoDjin Ansible playbooks for deployment on AWS as well as the rationale that lead to it. read more...

So far we had run the mail server with static hash files. As we add contributors and machines in the mix, it is about time to introduce a central account service (i.e. OpenLDAP). Since Rackspace and Amazon deprecated their first generation instances, re-building the mail infrastructure from scratch on a new system seemed appropriate - plus, it is fun! read more...

Ansible is a great piece of software to write IT automation scripts. The fact that Ansible is written in Python makes it even sweeter for us. read more...

For the past months, we have been using a workflow to communicate layout changes on the site that includes an in-browser screen capture, followed by a screenshot annotate step and concluded by a todo item in the backend database. read more...

Today we are going to mechanically upload DjaoDjin's requirements file to VersionEye through a git post-update hook. read more...

Database schema migration is a fact of life and South has imposed itself as the de facto solution within the Django community (at least until Django 1.7). The south docs are explicit, yet it is good to read the excellent south-explained post before diving in. read more...

We have worked as a cohesive team of Python developers for a long time. Either it is underscore variable names, spacing conventions or the 80-column rule, most of them had no problem. All the code written could be pretty much guaranteed to be PEP-8 compliant. read more...

In the popular series on how technical decisions are made, we will see Today why DjaoDjin picked django-jenkins, phantomjs, casperjs and django-casper to build its continuous integration infrastructure on. read more...

We've adopted Behave over Lettuce for our BDD needs here at Djaodjin. Below I'll present a summary of the motivations for this, and some criticisms of Behave. read more...

I wanted to setup my web app to authenticate through PAM as a general authentication mechanism. Since users are allowed to register and update password through the web app directly, I indented to use LDAP to hold user profile information. So I delve into setting-up PAM and LDAP. read more...

I setup redmine recently run through thin behind a nginx on a rackspace cloud machine. The interface is great and it seems like a very useful application if it was not so slow to respond. Simple http requests take forever even on a machine that experiences only minor traffic. read more...

Modx is a CMS system written in PHP. As a result, unless you install the not-yet-released PHP 5.4, you will need a PHP-enabled front web server. If you planned to use nginx you will have to do so through FastCGI (remember no built-in http server before PHP 5.4) which if, out-of-luck, you are running a PHP version below 5.3 will require a patch in the PHP source tree. Modx supports mysql as a database backend but there are no mention of postgresql. As a result, I have sticked with a "traditional" LAMP stack for now. read more...

In the most part I followed the ubuntu 11.04 openldap tutorial. The wikipedia article is also useful to understand some of the basics. I later stumbled upon LDAP for Rocket Scientists which definitely helped clarify some. read more...

Many times while validating a software product, you have to substitute a few parts in order to enable automated testing. This was again the case recently when I was confronted with a java application built as a jar file. The application relied on a complex subsystem of components that needed to be stubbed out in the test runner. read more...

It is kind of fun to look through your application logs and find traces of a hacker trying to break in. It might even be intellectually stimulating to play this game of hide and seek with another human being. Unfortunately most malicious attempts hitting your server will come from bots. Those don't get discouraged. Those don't change tactics. They keep trying to brute force passwords, even when you only allow private key login in your ssh daemon. They keep trying to access PHP scripts, even when you do not have any PHP stack running on your web server. Worse, if you allow people to leave comments on your web site, you are almost guarantee to attract spam bots that will waste precious bandwidth and mess up statistics you use to learn about your audience. read more...