Mail as Message Queue

Lets first understand, what is Message Queue (Credits: Lovisa Johansson)

A message is the data transported between the sender and the receiver application; it’s essentially a byte array with some headers on top. An example of a message could be something that tells one system to start processing a task, it could contains information about a finished task or just be a plain message.

Message queuing allows applications to communicate by sending messages to each other. The message queue provides a temporary message storage when the destination program is busy or not connected.

The basic architecture of a message queue is simple, there are client applications called producers that create messages and deliver them to the message queue. An other application, called consumer, connect to the queue and get the messages to be processed. Messages placed onto the queue are stored until the consumer retrieves them.

My Requirements

So, while developing my application, I was looking for a service, which acts like a Message Queue, however, my requirements were a bit different, I had some different types of data – Transaction Data, XML Response and some invoice like PDF files which should be generated and stored (for government compliance), all these items cannot be stored on the processing server, due to low disk availability and high risk.

My Environment

I have a server with 2 CPU and 8 cores, limited SSD disk and good bandwidth (5-10gbps, on Amazon EC2)

My Solution

Instead of using traditional Message Queuing applications like Amazon MQ, IBM MQ, I came up with a solution which suited my requirements, it was easy to implement and does not requires extra cents, so I decided to use IMAP/POP3 as my message broker. I had never used any MQ service with PHP, so going with any existing MQ means investing time learning it, I had tight deadlines to meet. The bottleneck was the generation of PDF document which should be generated on the time of transaction, it should be tamper proof (no text / csv files), digitally signed and should be stored for 5 years. (The authorities may ask for any random file and it should be compliant with there requirements). Client did not wanted to store or archive the same on cloud or S3.

What I did?

I installed DoveCot and PostFix on the same box (the box is not connected to Internet), configured the imap/smtp services and created an email account (mq@rajiv.local) with 5GB of inbox size.

In my application, instead of sending the required data and related files to MQ, I sent an email to mq@rajiv.local, I attached the generated files as email attachments, created a timestamp string to identify the message, used different custom email fields to feed in data which will help me to distinguish the email or message and I am done.

The retrieval of messages was really simple, I simply used a script to connect to the server using Imap (143) and process messages one by one, all the variables setup in mail headers and message body were stored in database and the files were moved to the archive server through FTP, I used Cron to execute this script every 180 seconds.

Since everything was happening on localhost, there was little or zero latency, later on, I moved to a dedicated mail server and created different email accounts for different applications or processes.

Benefits of Imap over MQ

There were a few, MQ is used for small messages, my data was around 2mb-15mb, I was not sure MQ supports different data types or Files, but my solution did, also, with the Email ID, I could forward the same message to different email addresses, infact I did (to Gmail :D) for archival and future use.

If you would like to explore my solution or if you think my solution may fit in your requirements, feel free to contact me.

Kubernetes on Windows

I came across an article from Microsoft, which confirms, that with Kubernetes version 1.14, we can run Windows Containers on our Kubernetes Clusters.

This does not means that you can run Windows Containers on a Linux based Kubernetes host, we still need a Windows Server where we can run Windows based containers.

Windows Server 2016 and Windows Server 2019 can run Kubernetes, its sad to see that Windows Server 2012 is not supported, so if you are spinning up a Windows Server node or instance, make sure you go the right version, my recommendations would be Windows 2019.

Bitcoin Hacker Scam

From past few weeks, I have been receiving emails from unknown persons, claiming that they have hacked my router and caught me watching porn websites, and recorded live videos from my webcam, the persons are asking for money through Bitcoin.

Earlier I used to receive such emails as Text, I created an email filter with the keywords mostly used, it stopped for a while, but it started coming back again, at first I could not understood how did the email pass through my filters, but later I noticed that the email message was being sent as an Image, white background was used, so it was hard to detect.

I searched through google about this email and I see many people had reported that they had received similar emails, the from and to email addresses were same, so I updated my spam settings, and it stopped such emails.

I had stored all such emails received and I was going through them, I noticed that while all emails had the same message, but the bitcoin address was different, I also checked the email headers and I found that all emails were being sent from email servers from different locations, some being compromised server and sometimes using free email providers.

This is a great example of social hacking, emails are being sent out to masses, asking ransom through bitcoin, so mostly all people who believe this email would signup with Bitcoin and purchase some coins to send. I also investigated whether the given bitcoin address had received any funds, and I did not find any transactions happening on these bitcoin wallets.

I also received emails which contained my email address and password, but these password were being used by me when I was a student, and I used simple passwords, it was back in 2000, when websites stored passwords as plain text, so if you are using passwords which you had used before 2005, kindly change your password and make it a random, complex sequence.

Oracle Java is no longer free

I was trying to install Oracle Java 8, in one of my projects today and I found that Oracle would not install on my Linux operating system, (I am using Ubuntu 18.04, trying to install through apt-get).

sudo apt-get install oracle-java8-installer
Reading package lists… Done
Building dependency tree
Reading state information… Done
Package oracle-java8-installer is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package ‘oracle-java8-installer’ has no installation candidate

So I tried to find a solution, visited various websites, but none of the suggested methods worked, later, I landed on this page –

https://launchpad.net/~webupd8team/+archive/ubuntu/java

Here, it is clearly said that support for Oracle Java PPA Installer is discontinued, from April 16, 2019, as Oracle Java is Free for personal and development use, but you need to have an active subscription (paid) to continue using Oracle Java for commercial purpose.

Here are the subscription options, please note that price and product availability may vary as per your country, the prices below are for US –

Oracle Java SE Desktop Subscription
1 Year Term Subscription that combines Java SE Licensing and Support for use on Desktops, Servers or Cloud deployments.
US$15.00 – US$30.00
Oracle Java SE Subscription
1 Year Term Subscription that combines Java SE Licensing and Support for use on Desktops, Servers or Cloud deployments.
US$150.00 – US$300.00
Oracle Java Development Tools Support
Support for NetBeans, Oracle JDeveloper and Oracle Enterprise Pack for Eclipse.
US$1,200.00

If you don’t want to pay for using Java, then you need to switch to Open JDK, its available for free for both commercial and personal use.

Remote Employee – Docker, AWS, Kubernetes, VMWare, Hosting

If you are looking for a remote employee on the following technologies, then feel free to hire me.

Devops – Docker, Kubernetes

CICD – Jenkins, Ansible, Terraform

Cloud – Amazon EC2, Linode, Azure, IBM

Hosting – Windows, Linux, CPanel, Helm, Plesk

Virtualization – VMWare, HyperV

Programming – PHP, ASP.Net, C#

In past, I have worked as a Level 3 support for various hosting environments, I have also developed SaaS Applications using PHP and .Net framework, I know Laravel, CakePHP and CodeIgnitor.

I am available from UTC 3 AM to UTC 3 PM, I can maintain your hosting environment, solve tickets, provide L2/L3 support for various platforms.