Commit notifications for Amazon CodeCommit using a Lambda Function and a Telegram Bot

Telegram is one of my favorite chat applications – it provides security, super speed, and a myriad of other features that you don’t find in almost any other chat service. Another such service is their Bot framework, which allows the creation of chat bots for Telegram that can do so many different things. Some of these are bots for services such as Gitlab and Bitbucket, which work off of webhooks to send commit details for repositories to a given chat on Telegram.

Amazon Web Services’ CodeCommit is a managed version control service provided by AWS, which can be configured to use Git as its underlying platform. When using CodeCommit, I wanted to send notifications on new commits to a Telegram group similarly to what the aforementioned bots for Gitlab and Bitbucket do. So I set out to achieve this using triggers from CodeCommit, a Lambda function and a very simple Telegram bot.

On a push to a given repository, CodeCommit will invoke a trigger which calls an AWS Lambda function, which in turn runs a RESTful web service call to the Telegram Bot API

As seen above, I created a Lambda function which is triggered by specific events on the CodeCommit repository, which contains the code needed to send a RESTful GET request to the Telegram Bot API to send a message to a specific group using the Telegram Bot.

So, let’s get down to brass tacks:

  1. Create the Telegram Bot: All you need to do is chat with @BotFather on Telegram. This is a Telegram Bot that helps you create new bots. Through some simple commands, you can get a new bot created for yourself and receive an auth token generated as well:
    Chat with BotFather to get your bot created

    NOTE: While @BotFather insists that it may come back some day to me with a request of its own, it has not done so yet, so I’m hoping I won’t be finding any severed horse heads on my bed anytime soon.

  2. Assuming you already have an AWS CodeCommit repository, create your Lambda function: You can do this the other way around as well, but I prefer to create the Lambda function and then bind the CodeCommit repository to it, as AWS lets you do this very easily. You can specify which repository you want to work with, what to name the trigger etc on the create trigger workflow on the Lambda function:
    Drag and drop a CodeCommit trigger from the left
    Then choose your repository, name your trigger etc below

    Once you’ve configured this stuff, you can go ahead and…

  3. Code your Lambda function: Here’s the sample code:
    var http = require('https');
    var AWS = require('aws-sdk');
    
    exports.handler = (event, context) => { 
        var codecommit = new AWS.CodeCommit({ apiVersion: '2015-04-13' });
        
        // Build Telegram Bot URL
        var baseUrl = "https://api.telegram.org/MY_BOT_TOKEN/sendMessage?chat_id=MY_CHAT_ID&text=";
        
        // Get the commit ID from the event
        var commits = event.Records[0].codecommit.references.map(
            function(reference) {
                return reference.commit;
            });
        console.log('CommitId:', commits);
        
        // Get the repository from the event and use it to get details of the commit
        var repository = event.Records[0].eventSourceARN.split(":")[5];
        codecommit.getCommit({
            commitId: commits[0],
            repositoryName: repository
        }, function(err, data) {
            if(err) {
                context.fail(err);
            } else {
                console.log(data);
                var commit = data.commit;
                var commitDetails = 'New commit to my repo: \nRef: ' + event.Records[0].codecommit.references[0].ref 
                    + '\n' + 'Message: ' + commit.message + 'Author: ' + commit.author.name + ' <' + commit.author.email + '>';
                var url = baseUrl + commitDetails;
                http.get(url, function(res) {
                    console.log("Success");
                    context.succeed();
                  }).on('error', function(e) {
                    console.log("Failed");
                    context.fail();
                  });
            }
        });
    };
    

    I wanted to send my messages to a specific Telegram group so I used @RawDataBot to get the group ID of that group, which is basically the group where the team members who work on this particular code repository are. @RawDataBot will give you a massive JSON string as soon as it joins the group, in which the chat ID will be included.

    In the code above, I’ve used the aws-sdk npm package to extract the CodeCommit JS API, which can be used to extract things such as commit details from the minimal information that is provided to you by the CodeCommit trigger. Actually, all you can get out of the CodeCommit trigger (the event object that is passed to the Lambda function) is the Ref of the repository that the commit occurred on and the commit ID.

    Then some mediocre object manipulation later, I’m making a very simple http call to the Telegram Bot API endpoint, which in turn invokes my bot to send a message to the group I have specified. In the end, it looks something like this:

    There’s a lot going on here. We have the bot sending messages about commits as well as builds (through a CodePipeline trigger, not covered in this post), and there’s also some gloating from me on some other conversation we were having on the group ūüėÄ

     

    And well, that’s how it’s done.

A boilerplate project for NodeJS + ExpressJS on ECMAScript 6 with MongoDB

With the advent of JavaScript ECMAScript 6/ECMAScript 2016/ES6, a whole bunch of new features were introduced, most of them being game-changers for anyone who wanted to switch over. My colleague Ragavan and I took it on ourselves to convert one of our existing ES5 NodeJS projects to ES6, and I thought it would be good to put together a base project that anyone could use to bootstrap a typical NodeJS + ExpressJS + MongoDB + REST project using the tools that we used.

We have used the following tools to make this work:

The full code can be found on Github. A huge shout out to Ashantha Lahiru who worked really hard to make the code presentable and more generic, as well as the MongoDB integration.

 

Troubleshooting Blazemeter incompatibility issues

Blazemeter is an awesome wrapper for JMeter that allows you to run JMeter load tests from various locations and generate awesome reports.

One of the coolest features of Blazemeter is that it allows you to upload existing JMeter Test Suites in the form of JMX files to execute those tests with any load that you specify.

However, I found that startup itself was failing on Blazemeter when I uploaded a JMX file generated by JMeter 3.1. Upon consulting the helpful support team, I was told that their compatibility with JMeter 3.1 is experimental, and I should remove a bunch of listeners I had added for reporting from the JMeter file.

This solved the problem, and I was able to run my tests.

Pokémon Go is here!

Now this isn’t necessarily a post on programming, but I’m so excited about this new game that I just had to post somewhere.

As an ardent fan of Ingress, I was very excited when they announced that Niantic Labs who built Ingress would be taking one of my favorite franchises, Pokémon, and building an AR game based on the same principles.

Managed to download and install the Beta testing version of the game today, and try it out. So here are my first impressions:

  • The game follows the same premise as the beloved games from GameFreak; there’s a Professor who gives you a started¬†Pok√©mon (Squirtle, Charmander or Bulbusaur) and sends you on your way to catch¬†Pok√©mon all around the world. That’s where the similarities end.
  • You find yourself in an AR world based on Google Maps and very heavily on the locations tagged as portals in Ingress: Some of them are Gyms (You can’t do anything at a gym until you’re Level 5 as a trainer; I’m still Level 4 so don’t know what’ll happen here), and the others are Pokestops, which follow the hacking mechanism in Ingress and give you items when you’re in range; although the item spawn doesn’t seem to be randomized like in Ingress: the Pokestop next to me right now just gives 3¬†Some Ingress portals are Gyms, others are Pokestops which follow the hack mechanism in Ingress and give items (the item spawn doesn’t seem to be randomized though, the Pokestop right next to me seems to only give 3¬†Pok√© Balls).
Some Ingress portals are Gyms, others are Pokestops
Some Ingress portals are Gyms, others are Pokestops
  • So I chose Charmander as my starter¬†Pok√©mon (ah, the nostalgia!).¬†Pok√©mon show up on the¬†Pok√©dex and the¬†Pok√©mon menu, and seem to have a few stats:
Charmander!
Charmander!
  • So there are two options, Power Up and Evolve. Each¬†Pok√©mon gives you candy for that type of¬†Pok√©mon when you catch it (and presumably, when you battle with it). I managed to catch enough Pidgeys to evolve it to Pidgeotto (needed 12 Pidgey candies):
Pidgey evolves!
Pidgey evolves!
Pidgeotto
Pidgeotto

 

 

 

 

 

 

 

 

 

 

 

  • Evolving my Pidgey seemed to give it new moves but did not improve its stats, which was rather disappointing. CP, or Combat Points, seem to be needed for battles as explained here. I can’t battle until I’m Level 5 though, so I don’t know yet.
  • Catching a¬†Pok√©mon is pretty exciting, it turns on the camera and shows the¬†Pok√©mon on screen where you have to throw¬†Pok√© Balls at it by flicking balls towards them. You can actually miss: I ended up throwing about 10¬†Pok√© Balls at a pesky Zubat.
Catching Pokémon in bed
Catching Pokémon in bed

Catching Pokémon gives you trainer experience too! And I managed to level up to Level 4:

Level 4!
Level 4!
  • Like in the console games, you can use Incense to attract¬†Pok√©mon to where you are; this didn’t seem as effective as the incense in the console games tho, I only got two extra¬†Pok√©mon and the spawn rate doesn’t seem to get increased much.
  • There is also an Egg mechanic where you get eggs to carry around for a certain distance and then they will hatch:
Eggs
Eggs (screenshot credit: https://kulendra.net)
  • There’s also the good ol’ Trainer Card (which tracks Trainer Stats) and Journal (which tracks activity):
    Journal
    Journal
    Trainer Card
    Trainer Card

     

 

 

 

 

 

 

 

 

 

 

  • There is also a store where you can buy items: Monetization strategy! ūüėÄ

 

Pokémon Store
Pokémon Store

And that wraps up my first impressions on the game. Will post more once I can actually get in to a battle with someone. Until then, Gotta Catch ’em All!!!

Running Cloudant Queries through the Cloudant NPM Package

Cloudant is a recently-popular solution for cloud-based NoSQL databases. It is based heavily on CouchDB, and provides a very easy-to-use HTTP verb-based web service interface to carry out database operations.

When using Cloudant with Node.js or Express.js, the Cloudant NPM Package, which is basically a wrapper for the CouchDB NPM package known as nano, comes in handy. But while their documentation states how to execute various operations such as getting a document by its ID, doing bulk operations etc, it is quite obscure on how to execute the extremely useful operations based on Cloudant Query, which allows you to write complex selectors like the following:

 

{
  "selector": {
    "_id": "myid",
    "$or" : [
        {
          "$and" : [
            {
                "endDate" : {"$gt" : "2015-11-05"}
            },
            {
                "endDate" : {"$lte" : "2015-11-30"}
            }
          ]
        },
        {
          "$and" : [
              {
                  "startDate" : {"$gte" : "2015-11-05"}
              },
              {
                  "startDate" : {"$lt" : "2015-11-30"}
              }
          ]
        }
    ],
    "usertype":"admin"
  }
}

So how do you execute a query like that through the provided functions in the npm package for Cloudant? The secret lies in the find function provided in the package. The above could be executed as;

 


db.find({"selector": {
    "_id": "myid",
    "$or" : [
        {
          "$and" : [
            {
                "endDate" : {"$gt" : "2015-11-05"}
            },
            {
                "endDate" : {"$lte" : "2015-11-30"}
            }
          ]
        },
        {
          "$and" : [
              {
                  "startDate" : {"$gte" : "2015-11-05"}
              },
              {
                  "startDate" : {"$lt" : "2015-11-30"}
              }
          ]
        }
    ],
    "usertype":"admin"
  }
}, function(error, result) {});

And simple as that, you can execute any complex query that works on Cloudant Query using the Cloudant npm package.

GSoC 2015 – Moorsp Plugin for Moodle – Wrap Up

It has been a hectic few months as a Google Summer of Code student for 2015 for Moodle, and it has come to a successful conclusion.

It was my greatest pleasure to work on Moodle throughout this period, to get to know and respect my awesome mentor, Dan Marsden, to learn about how Moodle and its community functions, and to eventually be able to help that community with a successfully completed GSoC project.

I set out to develop a skeleton plugin known as Moorsp for Moodle’s Plagiarism Framework, to incorporate the latest and greatest of Moodle’s framework goals within a testable plugin that wouldn’t need commercial logins to run automated tests. At the end of the project, all my code has been successfully integrated in to the Moorsp base code¬†, and I have been awarded a pass by the Moodle community, both in terms of my contribution in code and community engagement. This makes me extremely happy.

I hope to continue my work on Moodle and have already started to help Dan with integrating some of the newer concepts in the Moodle framework in to some of the older plagiarism plugins such as the Urkund Plagiarism Plugin which is maintained by Dan himself. I believe that the most important part of me doing a GSoC project is gaining the ability to integrate and work closely with the Moodle community, an opportunity which I absolutely will not let go to waste.

Finally, I will leave you with the lovely Moodle GSoC 2015 Student Badge awarded to me by moodle.org. Thank you, Moodle, for this lovely token of appreciation. I shall always cherish it.

Moodle_GSoC_student_2015

GSoC 2015 ‚Äď Moorsp Plugin for Moodle ‚Äď Update 11

This is my first update on the project in 2 weeks – mainly due to the fact that I have completed the bulk of the work required for the project and am now engaged in minor documentation and bug fixing tasks.

I brought the PHPUnit tests up-to-speed to be compatible with some of the changes I introduced to Moorsp during the Behat test writing phase – most notably that of a div being loaded along with submission content in the assignment module.

I had a discussion with Dan, my mentor, regarding sending a pull request to the main repository for Moorsp, and wish to merge the 44 extra commits listed here.

I also updated the README.md on the repository to give a guideline on Behat testing and unit testing Moorsp.

Moorsp README file

GSoC 2015 – Moorsp Plugin for Moodle – Update 10

The 10th week of my GSoC project for Moodle passed with me writing more Behat tests for the Moorsp plugin, completing the Behat test writing phase of development.

During writing tests I found some interesting bugs that were present in the code of Moorsp, most notable being that in onlinetext submission in the assign plugin, the content to be evaluated for plagiarism was being returned with <div class=”no-overflow”> tags around them, causing the filehashes to be mismatched and those text submissions not to be evaluated. I managed to build a workaround for this by hashing the contents with the offending tags included for onlinetext submissions.

Behat tests are running smoothly, with the main Behat tests implemented here.

Here is a sample of the final Behat test, show_teacher_plagiarism_status.feature, in action:

 

Behat tests in action

GSoC 2015 – Moorsp Plugin for Moodle – Update 9

The past week was spent continuing writing Behat tests for the Moorsp plugin. During writing the test for the feature where students are shown the plagiarism status on their submissions, I discovered that students were not being shown their plagiarism status even if the course manager had set a particular assignment, workshop or forum to show students this information. So I was able to fix this issue thanks to writing tests.

GSoC 2015 – Moorsp Plugin for Moodle – Update 8

The past week saw me reconfiguring my complete Moodle instance to get Behat testing working again, and the beginning of writing Behat tests for the Moorsp Plagiarism Plugin. I began writing tests for the following functionality:

  1. Enable/Disable Moorsp plugin – Completed
  2. Enable/Disable Moorsp submissions for assign – Completed
  3. Enable/Disable Moorsp submissions for workshop – Completed
  4. Enable/Disable Moorsp submissions for forum – Completed
  5. Show student plagiarism disclosure when submitting – Completed
  6. Show plagiarism level (whether plagiarised or not) on grading screen
  7. Show/hide plagiarism info from student

I will continue to write Behat and PHPUnit tests for the plugin next week.