Commit notifications for Amazon CodeCommit using a Lambda Function and a Telegram Bot

Telegram is one of my favorite chat applications – it provides security, super speed, and a myriad of other features that you don’t find in almost any other chat service. Another such service is their Bot framework, which allows the creation of chat bots for Telegram that can do so many different things. Some of these are bots for services such as Gitlab and Bitbucket, which work off of webhooks to send commit details for repositories to a given chat on Telegram.

Amazon Web Services’ CodeCommit is a managed version control service provided by AWS, which can be configured to use Git as its underlying platform. When using CodeCommit, I wanted to send notifications on new commits to a Telegram group similarly to what the aforementioned bots for Gitlab and Bitbucket do. So I set out to achieve this using triggers from CodeCommit, a Lambda function and a very simple Telegram bot.

On a push to a given repository, CodeCommit will invoke a trigger which calls an AWS Lambda function, which in turn runs a RESTful web service call to the Telegram Bot API

As seen above, I created a Lambda function which is triggered by specific events on the CodeCommit repository, which contains the code needed to send a RESTful GET request to the Telegram Bot API to send a message to a specific group using the Telegram Bot.

So, let’s get down to brass tacks:

  1. Create the Telegram Bot: All you need to do is chat with @BotFather on Telegram. This is a Telegram Bot that helps you create new bots. Through some simple commands, you can get a new bot created for yourself and receive an auth token generated as well:
    Chat with BotFather to get your bot created

    NOTE: While @BotFather insists that it may come back some day to me with a request of its own, it has not done so yet, so I’m hoping I won’t be finding any severed horse heads on my bed anytime soon.

  2. Assuming you already have an AWS CodeCommit repository, create your Lambda function: You can do this the other way around as well, but I prefer to create the Lambda function and then bind the CodeCommit repository to it, as AWS lets you do this very easily. You can specify which repository you want to work with, what to name the trigger etc on the create trigger workflow on the Lambda function:
    Drag and drop a CodeCommit trigger from the left
    Then choose your repository, name your trigger etc below

    Once you’ve configured this stuff, you can go ahead and…

  3. Code your Lambda function: Here’s the sample code:
    var http = require('https');
    var AWS = require('aws-sdk');
    
    exports.handler = (event, context) => { 
        var codecommit = new AWS.CodeCommit({ apiVersion: '2015-04-13' });
        
        // Build Telegram Bot URL
        var baseUrl = "https://api.telegram.org/MY_BOT_TOKEN/sendMessage?chat_id=MY_CHAT_ID&text=";
        
        // Get the commit ID from the event
        var commits = event.Records[0].codecommit.references.map(
            function(reference) {
                return reference.commit;
            });
        console.log('CommitId:', commits);
        
        // Get the repository from the event and use it to get details of the commit
        var repository = event.Records[0].eventSourceARN.split(":")[5];
        codecommit.getCommit({
            commitId: commits[0],
            repositoryName: repository
        }, function(err, data) {
            if(err) {
                context.fail(err);
            } else {
                console.log(data);
                var commit = data.commit;
                var commitDetails = 'New commit to my repo: \nRef: ' + event.Records[0].codecommit.references[0].ref 
                    + '\n' + 'Message: ' + commit.message + 'Author: ' + commit.author.name + ' <' + commit.author.email + '>';
                var url = baseUrl + commitDetails;
                http.get(url, function(res) {
                    console.log("Success");
                    context.succeed();
                  }).on('error', function(e) {
                    console.log("Failed");
                    context.fail();
                  });
            }
        });
    };
    

    I wanted to send my messages to a specific Telegram group so I used @RawDataBot to get the group ID of that group, which is basically the group where the team members who work on this particular code repository are. @RawDataBot will give you a massive JSON string as soon as it joins the group, in which the chat ID will be included.

    In the code above, I’ve used the aws-sdk npm package to extract the CodeCommit JS API, which can be used to extract things such as commit details from the minimal information that is provided to you by the CodeCommit trigger. Actually, all you can get out of the CodeCommit trigger (the event object that is passed to the Lambda function) is the Ref of the repository that the commit occurred on and the commit ID.

    Then some mediocre object manipulation later, I’m making a very simple http call to the Telegram Bot API endpoint, which in turn invokes my bot to send a message to the group I have specified. In the end, it looks something like this:

    There’s a lot going on here. We have the bot sending messages about commits as well as builds (through a CodePipeline trigger, not covered in this post), and there’s also some gloating from me on some other conversation we were having on the group ūüėÄ

     

    And well, that’s how it’s done.

A boilerplate project for NodeJS + ExpressJS on ECMAScript 6 with MongoDB

With the advent of JavaScript ECMAScript 6/ECMAScript 2016/ES6, a whole bunch of new features were introduced, most of them being game-changers for anyone who wanted to switch over. My colleague Ragavan and I took it on ourselves to convert one of our existing ES5 NodeJS projects to ES6, and I thought it would be good to put together a base project that anyone could use to bootstrap a typical NodeJS + ExpressJS + MongoDB + REST project using the tools that we used.

We have used the following tools to make this work:

The full code can be found on Github. A huge shout out to Ashantha Lahiru who worked really hard to make the code presentable and more generic, as well as the MongoDB integration.

 

Troubleshooting Blazemeter incompatibility issues

Blazemeter is an awesome wrapper for JMeter that allows you to run JMeter load tests from various locations and generate awesome reports.

One of the coolest features of Blazemeter is that it allows you to upload existing JMeter Test Suites in the form of JMX files to execute those tests with any load that you specify.

However, I found that startup itself was failing on Blazemeter when I uploaded a JMX file generated by JMeter 3.1. Upon consulting the helpful support team, I was told that their compatibility with JMeter 3.1 is experimental, and I should remove a bunch of listeners I had added for reporting from the JMeter file.

This solved the problem, and I was able to run my tests.

Running Cloudant Queries through the Cloudant NPM Package

Cloudant is a recently-popular solution for cloud-based NoSQL databases. It is based heavily on CouchDB, and provides a very easy-to-use HTTP verb-based web service interface to carry out database operations.

When using Cloudant with Node.js or Express.js, the Cloudant NPM Package, which is basically a wrapper for the CouchDB NPM package known as nano, comes in handy. But while their documentation states how to execute various operations such as getting a document by its ID, doing bulk operations etc, it is quite obscure on how to execute the extremely useful operations based on Cloudant Query, which allows you to write complex selectors like the following:

 

{
  "selector": {
    "_id": "myid",
    "$or" : [
        {
          "$and" : [
            {
                "endDate" : {"$gt" : "2015-11-05"}
            },
            {
                "endDate" : {"$lte" : "2015-11-30"}
            }
          ]
        },
        {
          "$and" : [
              {
                  "startDate" : {"$gte" : "2015-11-05"}
              },
              {
                  "startDate" : {"$lt" : "2015-11-30"}
              }
          ]
        }
    ],
    "usertype":"admin"
  }
}

So how do you execute a query like that through the provided functions in the npm package for Cloudant? The secret lies in the find function provided in the package. The above could be executed as;

 


db.find({"selector": {
    "_id": "myid",
    "$or" : [
        {
          "$and" : [
            {
                "endDate" : {"$gt" : "2015-11-05"}
            },
            {
                "endDate" : {"$lte" : "2015-11-30"}
            }
          ]
        },
        {
          "$and" : [
              {
                  "startDate" : {"$gte" : "2015-11-05"}
              },
              {
                  "startDate" : {"$lt" : "2015-11-30"}
              }
          ]
        }
    ],
    "usertype":"admin"
  }
}, function(error, result) {});

And simple as that, you can execute any complex query that works on Cloudant Query using the Cloudant npm package.

Running PHP applications in Apache Tomcat 7

Apache Tomcat 7 is a popular server for running Java applications – but say you have a unique instance where you want to run both PHP and Java applications on Tomcat but don’t want to run two separate servers (i.e. Apache HTTPD + Tomcat) to achieve it?

PLEASE NOTE THAT THIS METHOD DOES NOT WORK ON TOMCAT 8 AS OF THE DATE OF WRITING.

For this we’re going to use the PHP – Java Bridge for Tomcat, which is a handy little package that will sort out PHP execution on top of Tomcat.

The first step is to download the Javabridge template from this link at SourceForge.¬†This will give you a WAR file that you can directly drop in to your Tomcat server’s webapps folder.

For Windows users, the next step is to download PHP and extract it somewhere:

php

 

For Linux users, just install php5 using apt-get, yum, or your preferred package management tool.

Next step is to add PHP to the PATH variable. Append the path to your PHP folder to the end of your respective PATH variable.

Next, drop the JavaBridgeTemplate61.war file in to your Tomcat server’s webapps folder and start Tomcat.

When you navigate to your server’s Java Bridge folder (this is http://localhost:8080/JavaBridgeTemplate61/ by default), you’ll be able to see the Java Bridge’s index page:

Javabridge welcome page

 

If you go to javabridge/test.php, you’ll be able to see the output from the phpinfo() function:

phpinfo

 

If you open up test.php in the JavaBridgeTemplate folder in your Tomcat’s webapp folder, you can edit some of the code:

test

hello

 

 

Now you can deploy any PHP application within the JavaBridgeTemplate folder and rename the folder to whatever you want. You can add many folders like this for different PHP applications.

Note that the JavaBridge runs PHP as CGI. If you want to enable/disable specific PHP extensions you can simply enable them as you would do normally on your php.ini file.

Behat testing PHP applications with Moodle as an example

The Behat testing framework  advocates the relatively-new concept of Behavior-Driven Development (BDD)/Acceptance-Test Driven Development (ATDD), where human-readable tests are written for highly user-oriented tasks by the developers themselves. This automates the User Acceptance Testing (UAT) process to a certain degree as the tests themselves are not written in highly-technical terminology and follow front-end testing paradigms. Essentially, the idea is that you define a test on how your application should work from the front-end, and then develop that feature from there. Behat works on PHP 5.3 and upwards.

Moodle is a Free and Open Source Learning Management Tool with an active community from all around the world. Moodle is also a good example for extensive use of Behat to test its features. (I should note that Sahana makes use of the Robot framework for ATDD as well)

I thought of covering some of the basics in the Behat framework using examples from how Moodle does it. While Moodle’s implementation might be somewhat different from other projects that use Behat, the basics of each implementation should be reasonably similar.

The following is an example for a typical Behat test, which Behat calls a feature file.

@auth
 Feature: Login
   In order to login
   As a moodle user
   I need to be able to validate the username and password against moodle

   Scenario: Login as an existing user
     Given I am on "login/index.php"
     When I fill in "username" with "admin"
     And I fill in "password" with "admin"
     And I press "loginbtn"
     Then I should see "Moodle 101: Course Name"

   Scenario: Login as a non-existent user
     Given I am on "login/index.php"
     When I fill in "username" with "admin"
     And I fill in "password" with "admin"
     And I press "loginbtn"
     Then I should see "Moodle 101: Course Name"

Notice that the feature is written in Gherkin,  which is a feature language taken from Cucumber, the ATDD framework for Ruby on Rails. These Gherkin commands are tied in to a PHP function on the Behat framework, for example,

And I press "loginbtn"

ties in to;

/**
 * Presses button with specified id|name|title|alt|value.
 *
 * @When /^I press "(?P&lt;button_string&gt;(?:[^"]|\\")*)"$/
 * @throws ElementNotFoundException Thrown by behat_base::find
 * @param string $button
 */
public function press_button($button) {

    // Ensures the button is present.
    $buttonnode = $this-&gt;find_button($button);
    $buttonnode-&gt;press();
}

in behat_forms.php in the Behat framework. It is also possible to re-use these functions to write very specific test functions for your application. For example, Moodle contains an ‘Editing mode’ on the Courses module which allows course administrators and teachers to see edit¬†links on various nodes within a course. ¬†Moodle’s implementation of Behat provides

And I turn editing mode on

which translates to

/**
 * Turns editing mode on.
 * @Given /^I turn editing mode on$/
 */
 public function i_turn_editing_mode_on() {
   return new Given('I press "' . get_string('turneditingon') . '"');
 }

which is implemented in the behat_course.php custom class within Moodle’s course core module. Note that get_string($string) is a function specific to Moodle’s framework.

An interesting feature of Behat is its integration with Mink and Selenium Webdriver to execute the tests on UI itself. Once you run the tests, Selenium will open up a browser window and execute the steps described in each feature file sequentially, while you can watch it being executed.

The Moodle community provides a very comprehensive guide on how to get started with Behat on Moodle, so I will not reproduce that here. I will get on to describing more of the finer points of Behat testing later on.

Adding the AndEngine Box2D Physics Extension to an AndEngine project

I’ve talked about AndEngine¬†before as a great framework for building 2D games on the Android platform. Today, I will be explaining how to include the AndEngine Box2D extension within an AndEngine project so that you can start implementing realistic physics within your game.

The AndEnginePhysicsBox2DExtension, provides an implementation of the popular C++ Box2D physics framework for AndEngine. Getting this doohicky integrated with your current AndEngine project can be a little tricky, however, and that’s what I’m going to explain today.

First, you need to have an Eclipse project with AndEngine included in any way you prefer. Note that you’ll have to have AndEngine as a library project in Eclipse before you begin, as the Box2D Extension will need it as well.

The next step is to clone the Github repository for the AndEnginePhysicsBox2DExtension.

Git Clone the Box2D Extension
Git Clone the Box2D Extension

Now, import the extension as a Library project to your Eclipse workspace. (Right click on workspace -> Import -> Android -> Android Project from Existing Code).

Import the extension as existing Android code to your workspace
Import the extension as existing Android code to your workspace

Once you’ve imported the extension, it will immediately build a set of JARs and various other binary files. Now, there are two ways to include the extension in to an existing AndEngine project:

  1. Include it as a library project (recommended) :
    1. Make sure your AndEnginePhysicsBox2DExtension workspace project is pegged as a library. You should be be able to check this by right clicking on the extension project, going to properties, and checking whether ‘Is Library’ is ticked under the Android section.
      Make sure that your extension project is a library
      Make sure that your extension project is a library

      Now, right click on your existing AndEngine project and go to Properties and go to the Android Section. In the libraries box at the bottom, you should be able to add the AndEnginePhysicsBox2DExtension as a library.

      Add the extension as a library to your AndEngine project
      Add the extension as a library to your AndEngine project
  2. Add AndEnginePhysicsBox2DExtension binaries to your libs folder:
    1. As I mentioned earlier, your AndEnginePhysicsBox2DExtension workspace project will build a set of extension binaries in the /bin folder as soon as you import it. Although it is not recommended by the developer of AndEngine, you can copy over these files to your AndEngine project’s /libs folder.
      Copy Physics Extension JAR files to your /libs folder
      Copy Physics Extension JAR files to your /libs folder

       

Now that you’ve included the AndEnginePhysicsBox2DExtension within your AndEngine project, you can proceed to write awesome physics code in to your game, like including the gravity of the first Death Star within the game’s environment!

Use the gravity of the first Death Star in your game! This is actually an Android thing, not an AndEngine thing.
Use the gravity of the first Death Star in your game! This is actually an Android thing, not an AndEngine thing.

Building and sending a SOAP Envelope via Axis2 Client

The Apache Axis2¬†library is one of the world’s most renowned FOSS solutions for writing and deploying web services. This library ties in very closely with the Apache Axiom¬†library which is used to build the Object Model needed to send and receive SOAP messages for web services built using Axis2.

Today we’re going to take a look at how to build a SOAP envelope from a String value and how to set it to an Axis2 service client message context. There are a few things we need to do in order to accomplish this:

  1. Create a ServiceClient object.
  2. Create an Options object and set the endpoint URL for the service we’re going to call.
  3. Create a new MessageContext object.
  4. Create a SOAPEnvelope object from a given string using Axis2 TransportUtils.
  5. Set the SOAP envelope object to the MessageContext object.
  6. Use the ServiceClient object’s createClient method to generate an OperationClient object for this operation.
  7. Add the MessageContext object to the OperationClient object.
  8. Execute the OperationClient object.

Code:

final String MY_OM_STRING = "<soapenv:Envelope xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\">\n"
                        + "   <soapenv:Header/>\n"
                        + "   <soapenv:Body>\n"
                        + "     <my_xml_tag>My Value</my_xml_tag>\n"
                        + "   </soapenv:Body>\n"
                        + "  </soapenv:Envelope>";
//1. Create new ServiceClient object
ServiceClient serviceClient = new ServiceClient();

//2. Create an Options object and set the endpoint URL for the service we're going to call
Options serviceOptions = new Options();
serviceOptions.setTo(new EndpointReference(ENDPOINT_URL);
serviceClient.setOptions(serviceOptions);

//3. Create a new MessageContext object
MessageContext messageContext = new MessageContext();

//4. Create a SOAPEnvelope object from a given string
OMElement myOMElement = AXIOMUtil.stringToOM(MY_OM_STRING);
SOAPEnvelope soapEnvelope = TransportUtils.createSOAPEnvelope(myOMElement);

//5. Set the SOAP envelope object to the MessageContext object
messageContext.setEnvelope(soapEnvelope);

//6. Use the ServiceClient object's createClient method to generate an OperationClient
OperationClient opClient = serviceClient.createClient(ServiceClient.ANON_OUT_IN_OP);

//7. Add the MessageContext object to the OperationClient object
opClient.addMessageContext(messageContext);

//8. Execute the OperationClient object
opClient.execute(true);

And that’s how its done.

Configuring and carrying out a FULLTEXT search in MySQL

When we normally have an application associated with a MySQL database and we want to carry out a search, we would normally go for the yucky “LIKE%” syntax when writing our search query. What if I told you that there is a much better indexed method built right in to MySQL that you could use to run an optimised and effective search? Welcome to the world of MySQL Fulltext searching. Today we’ll discuss how to define Fulltext indices on a MySQL database table and how to run a simple search query on them.

First things first, let’s install PhpMyAdmin. I’m going to assume that you already have MySQL installed. PhpMyAdmin is a handy tool that lets us manage our MySQL databases without having to go through the hassle of typing out long terminal commands. You can use any other similar GUI tool such as MySQL Workbench as well.

Now, I have a database called etutor where I have a table called questions. In order to set up Fulltext indices on this table, I’m going to have to convert it to a MyISAM-engine-based table. This can be achieved by selecting the table and going to the ‘Operations’ tab:

You need to change your database table’s engine to MyISAM

When I open up the structure of the questions table in PhpMyAdmin, there’s a small ‘fulltext’ icon on the right of each attribute of the table.

The Fulltext button can be seen next to each column name

This is the button with the little ‘T’ on it. Clicking on this will define that attribute (it needs to be VARCHAR or Text) type as a Fulltext index, which we can search on.

Expanding the ‘Details’ section at the bottom of the page will show you all the Fulltext indices you’ve defined on your table.

Its that simple to define a Fulltext index on your database table! Now let’s move on to some searching.

My search query will utilise the ‘MATCH’ and ‘AGAINST’ keywords in MySQL to search through my Fulltext indices:

SELECT * FROM questions WHERE MATCH(subject)
AGAINST('SEARCH_QUERY') OR MATCH(tags)
AGAINST('SEARCH_QUERY');

And frankly, it’s that simple to search through those pesky text indices too. You will find that there is a massive performance boost when using Fulltext search vs. the traditional ‘LIKE%’ query approach.

Some Physics functions in AndEngine GLES1.0 + Box2D

AndEngine is one of the best game engines available to build 2D games on the Android platform. It has a load of great features, one of which its integration with the Box2D physics engine. This is done via a great extension.

There are many sources out there to help developers with integrating physics features in to AndEngine, including examples from the author of the engine himself, however, I thought of writing my own helpful guide with some of the lesser-explained functions of AndEngine’s physics engine.

You’ll be able to use this awesome blog post by Maya Posch¬†to set up AndEngine and get it configured with the AndEngine Physics Box2D Extension. It’s a bit complicated, so if you’re lazy like me you could find the compiled sources from the previously-linked Examples repository (though this is NOT recommended).

Now, to get started on some of the functions that you’ll need. The Box2D environment, when oversimplified, contains the following main elements:

  • Physics World: This is where all the exciting business goes on, and where all the elements related to physics are stored and manipulated. Welcome to the world of physics!
  • Bodies: Bodies are similar to sprites in any game; they’re what move about, crash in to each other etc.
  • Vectors: These are what you’ll be applying on your bodies (you do know your basic physics, right?) to make them move around, rotate, stop, and every other conceivable thing.

Basics aside, we’ll look at a few functionalities that are rarely explained in most AndEngine physics examples:

  1. Contact Listeners:

Contact listeners are Box2D’s equivalent to collision detection in game engines. A ContactListener is defined for the Physics World in general, and will handle all contacts that happens in that world. That means, when a contact listener fires, you’ll only get the Fixtures (and underlying Bodies) of that particular contact to manipulate. Handling contact listeners in this way can get pretty dirty pretty fast, it turns in to a mess of checking fixtures for their underlying attributes and rechecking to see if they’re the actual body within the physics world that you want to check for collisions. Well, you did want to have smooth physics in your game, right? Now pay the price. ūüėČ

Following is a code snippet from a car racing game I worked on:

mPhysicsWorld.setContactListener(new ContactListener() {

			public void preSolve(Contact contact, Manifold oldManifold) {

			}

			public void postSolve(Contact contact, ContactImpulse impulse) {

			}

			public void endContact(Contact contact) {

			}

			public void beginContact(Contact contact) {

				if ( contact.getFixtureA().getBody().getUserData() != null &&
						contact.getFixtureB().getBody().getUserData() != null  ) {
                                        //I stored a 'Car' object in the userdata of each physics body as I created it
					final Car carA = (Car)contact.getFixtureA().getBody().getUserData();
					final Car carB = (Car)contact.getFixtureB().getBody().getUserData();
					if ( carA.getId() == 1 || carB.getId() == 1 ) {
						carState = CAR_STATE_CRASHED;
						vibrator.vibrate(100);
                                        }
                                }
                      }
});

By returning the bodies associated with each fixture in teh contact, I have managed to get the ‘Car’ type objects stored within the userdata elements within each body. By checking a specific value within these car objects, I have decided what to do next. Any contact handling that you’d do with AndEngine + Box2D would have to be a lot of if-else clauses where you check whether the bodies crashing together within the physics world are the bodies you want to check for collisions with. The presolve() an postsolve() methods are concerned with preparing the physics world for the contact, where you can set various attributes like whether contacts are enabled at that point.

2. Setting your body’s velocity to 0:

This might seem like a weird requirement, but believe me, after your sprites start bouncing around the screen like crazy ping pong balls, you’ll thank me. This is something it took me a while to figure out, so I thought I’d share:

Vector2 linearZero = Vector2Pool.obtain(0,0);
playerBody.setLinearVelocity(linearZero);
Vector2Pool.recycle(linearZero);//don't forget to recycle your vectors! It helps save the environment.

Ok, so you have your playerBody right? What you do is, you initialize a vector with 0 values, and you apply that as the linear velocity to your body. That will stop the body from moving on all plains.

3. Insta-move your sprite to any place on the screen:

Sometimes, even when you have a physics engine at your disposal, you just want to magically materialize your sprite somewhere on the screen. The problem is, if you use the normal AndEngine sprite.setPosition() method, the physics body you’ve attached to the sprite is not going to move with it. You need to use Box2D itself to insta-move your sprite (once you move the body, the sprite will always move with it, but not vice-versa):

 

Vector2 transformVector = Vector2Pool.obtain(newXPosition, newYPosition);
playerBody.setTransform(transformVector, playerBody.getAngle());
Vector2Pool.recycle(transformVector);

Vectors again. Simply build a new vector containing the X and Y coordinates you want to move your sprite to, and use the setTransform() method on the body attached to the sprite.

 

That’s it from me for now. More soon!