Pallab's Blog

I learn, I write.

Getting started with Cloud Foundry — March 12, 2017

Getting started with Cloud Foundry

For the past few weeks, I have been working exclusively with Cloud Foundry. However, my work is focused more on creating the Cloud Foundry platform for the developers to use rather than deploying my app on it. So, I decided to check out how an app is actually deployed on Cloud Foundry. In this post, I’ll list down the steps to successfully deploy a sample app on Pivotal Cloud Foundry. So, let’s get started.

Before we start running our commands on the console, let’s get some understanding about cloud computing services and where Cloud Foundry fits.

 

cloud-foundry

As you can see in the image above, there are three broad categories of cloud-based services. Examples of IaaS are OpenStack, Amazon Web Services (AWS) which provide you with the base infrastructure. Examples of SaaS are Google Apps, Office 365, etc.

PaaS is where Cloud Foundry fits in. The pre-requisite for Cloud Foundry is to have an IaaS in place on top of which you can deploy Cloud Foundry.

What is Cloud Foundry?

Cloud Foundry, in simple words, is a cloud computing platform which facilitates developers to build, run, deploy and scale their apps without having to worry about anything else.

It handles most of an application’s resource management and thus reducing the workload of the developers helping them focus more on the application development.

I’ll not be explaining Cloud Foundry’s architecture, its components, etc. since the post is more about getting you up to speed with app deployment.

Creating a Pivotal Web Services account

Alright, let’s begin now. The very first thing that I want you to do is to go to http://run.pivotal.io/ and sign up for a free account. You won’t be asked for any credit card details but, Pivotal requires you to verify your phone number. Also, during the account creation, you may be prompted to enter an org name. Simply enter -org if you’re not able to come up with another name.

Once you’re done, please log in using your credentials and then come back here.

Setting up the CF Command Line Tool

Good, now that you have set up your PWS account, you’re going to need something called the cf cli. So, follow this link and set it up on your OS https://docs.cloudfoundry.org/cf-cli/install-go-cli.html. It’s really easy and straight forward to set it up, just follow the guidelines in the link.

Cool, once you have installed it, open a terminal or command prompt and run the following command,

$ cf -v
cf version 6.24.0+c7a32d3.2017-02-10

If you see something similar on your screen, then we’re good.

Next step is to log in to your account from the command line. Type the following command,

$ cf login -a https://api.run.pivotal.io

You will be prompted to enter your email and password.

$ cf login -a https://api.run.pivotal.io
API endpoint: https://api.run.pivotal.io

Email> pallabkumarpain@gmail.com

Password> 
Authenticating...
OK

Targeted org pallab-org

Targeted space development

Use the same email and password that you had used to create an account on PWS

Alright, once you’re logged in, we’re ready to deploy our app.

Deploying an application and creating application manifest

We’ll be deploying a sample application which you can get from here https://github.com/pallabpain/userStory

Download the app on your system and navigate to the app’s folder from the console. Cloud Foundry requires you to have a ‘manifest.yml’ inside your app’s root directory which it reads to get app-related settings. So, create a file named manifest with a .yml in your app’s folder. Your folder structure should look something like this.

.
├── app
├── config.js
├── manifest.yml
├── package.json
├── public
├── README.md
└── server.js

Now open the manifest file and add the following lines to it and then save it.

---
applications:
- name: userStory
  memory: 128MB
  instances: 1

Let’s understand what the above lines mean. The manifest.yml dictates the app configuration which Cloud Foundry should use when it is being deployed.

  • name: Name of your app
  • memory: Amount of memory you want to allocate for your app
  • instances: Number of instances of the app that should be running

Let’s deploy our app now. Run the following command on your console. Make sure that you’re in your app’s directory where the manifest is located.

$ cf push

Cloud Foundry will now deploy your app. You will have to wait a little and let Cloud Foundry perform the tasks. Once the app is deployed successful, you will see something similar on your console.

requested state: started
instances: 1/1
usage: 128M x 1 instances
urls: userstory-1.cfapps.io
last uploaded: Sat Mar 11 09:16:39 UTC 2017
stack: cflinuxfs2
buildpack: node.js 1.5.30

   state   since                  cpu   memory        disk        details
#0 running 2017-03-11 02:47:23 PM 17.2% 97.8M of 128M 59.3M of 1G

If you’re getting the following error:

Server error, status code: 400, error code: 210003, message: The host is taken: userstory

change the name of the app in the manifest to something unique and run cf push again.

Now, copy the link to the app from the urls field and open it in your browser. You should see the landing page of the app. You can try signing up, but at this moment it isn’t going to work since we have not configured MongoDB database for the application to use. Let’s get our app working in the next section.

Creating a service

Since our application requires MongoDB, let’s add a MongoDB service from the marketplace. Run the following command:

$ cf marketplace

Screenshot from 2017-03-15 07-46-57

You’ll be shown a list of services that are available for you to consume. Of course, some of them require you to pay for you to use them. We require the mlab service which is free to use. Run the following command on your console to create an instance of the mlab service:

$ cf create-service mlab sandbox mymongo

create-service is the command for creating a new service instance. mlab is the name of the service. sandbox is a plan provided by mlab which we are using and mymongo is the name of our service instance.

Once the service instance is created, run the following command:

$ cf services

name      service   plan      bound apps   last operation
mymongo   mlab      sandbox                create succeeded

You should a similar output.

Now that we have created a service instance, let’s see how we can make our app consume it.

Binding a service to your app

Open your manifest file again, and this time add two more lines to it as shown below:

---
applications:
- name: userstory-1
  memory: 128MB
  instances: 1
services:
- mymongo

Run cf push again. Although we have added the service in our manifest, we still have to make some changes in our app’s code such that it is able to use the service. Once your app is running, run the following command:

$ cf env <your-app-name>

Running the above code will give you the environment variables for your app. Look for VCAP_SERVICES in the json output. You should be able to find the mlab service inside it. Here’s how it looks on my console,

{  
   "VCAP_SERVICES":{  
      "mlab":[  
         {  
            "credentials":{  
               "uri":"mongodb://CloudFoundry_3350q0fd_g859c87d_jcqbt1gq:qnkhH8cBkYcZ4xvvLBJ41VPVDshMJd8n@ds139327.mlab.com:39327/CloudFoundry_3350q0fd_g859c87d"
            },
            "label":"mlab",
            "name":"mymongo",
            "plan":"sandbox",
            "provider":null,
            "syslog_drain_url":null,
            "tags":[  
               "Cloud Databases",
               "Developer Tools",
               "Web-based",
               "Data Store",
               "document",
               "Windows",
               "Security",
               "IT Management",
               "mongodb",
               "Mac"
            ],
            "volume_mounts":[  

            ]
         }
      ]
   }
}

The app requires the uri to be able to connect to the database. So, we’ll make the changes in our code now.

Open the config.js file. You will see the following value in the current code,

"database":"mongodb://localhost:27017/userstory"

Your new config should look like the following:

var vcapServices = require('vcap_services');
var credentials = vcapServices.getCredentials('mlab');

module.exports = {
   "database": credentials.uri,
   "port": process.env.PORT || 3000,
   "secretKey": "aquaSurfer"
}

We just have to make one more change, add the following line to your package.json file.

"dependencies": {
   "vcap_services": "*",
   "bcrypt-nodejs": "0.0.3",
   "body-parser": "^1.14.1",
   "express": "^4.13.3",
   "jsonwebtoken": "^5.4.1",
   "mongoose": "^4.2.7",
   "morgan": "^1.6.1",
   "socket.io": "^1.3.7",
   "socketio": "^1.0.0"
 }

Save the changes and run cf push again. Once the app is running, open it in your browser and see if you can sign up. It should work this time around.

imageedit_2_4913641333

 

Scaling your app

Scaling is pretty easy with Cloud Foundry. There are two kinds of scaling,

  1. Horizontal, where you scale up or scale down the number of instances of the app.
  2. Vertical, where scale up or scale down the disk and the memory for your app.

The command for horizontal scaling is,

$ cf scale <your-app-name> -i <num_instances>

The commands for vertical scaling are,

$ cf scale <your-app-name> -m <memory_size>
$ cf scale <your-app-name> -k <disk_size>

I’m not going to explain this is detail as the post is getting longer. Instead, I’ll leave this link for you to check out https://docs.cloudfoundry.org/devguide/deploy-apps/cf-scale.html

Deleting your app

Since we deployed an app on Cloud Foundry, let’s also see how we can delete the app as well.

$ cf delete <your-app-name>

As simple as that.

Next Steps

If you have made it till here, congratulations! You’re now all set to go and explore more about Cloud Foundry. Check their documentation https://docs.cloudfoundry.org/#read-the-docs for more details. If you’re facing any issue, they have their own slack channel https://cloudfoundry.slack.com which is pretty active.

Advertisements
Working with JSON in Python — September 18, 2016

Working with JSON in Python

In this post, I’ll be exploring Python’s JSON module. I would suggest you go through json.org to get a better understanding of JSON if you’re not already aware of it.

Working with JSON is very simple in Python. Python already comes bundled with the json module which we would be seeing in this post.

The very first thing that we would do is, import the json module. Now, you can open the python interpreter or write it in a script or use some online coding platform to do it. I’ll leave the choice entirely up to you. So, let’s start.

import json

That should import the json module for you and we are good to go ahead now.

Converting from JSON to Python Object

The json module parses JSON either into a dictionary or a list in Python. To do that, it provides us with the load() and loads() methods. The difference between the two methods is that the former takes a file object as a parameter and the latter takes a JSON string as parameter.

import json
movies = '[{"title": "Despicable Me", "year": 2010, "watched": true}, {"title": "The Conjuring", "year": 2013, "watched": false}]'
json_to_python = json.loads(movies)

If you type print json_to_python, you should get the following output:

[{u'watched': True, u'title': u'Despicable Me', u'year': 2010}, {u'watched': False, u'title': u'The Conjuring', u'year': 2013}]

You can have the same JSON in a file and use the loads() method to convert it to a Python list like we did in the code above. Let’s see how we can do that.

import json
file_object = open("movies.json", "r")
json_to_python = json.load(file_object)

You should get the same output as you got above.

Converting from Python to JSON

In the earlier section, we covered how to convert JSON into a Python object. We’ll now see how we can do the other way round i.e. convert a Python object to JSON

import json
movies = [{'name': 'Finding Nemo', 'year': 2003, 'watched': True}, {'name': 'Fight Club', 'year': 1999, 'watched': False}]
python_to_json = json.dumps(movies)

If you print python_to_json, you will get the following JSON data:

[{"watched": true, "name": "Finding Nemo", "year": 2003}, {"watched": false, "name": "Fight Club", "year": 1999}]

And like we have load() for reading JSON data from a file, we have dump() for writing JSON data into a file and it’s fairly simple to write the code for it.

import json
file_object = open("movies.json", "w")
movies = [{'name': 'Finding Nemo', 'year': 2003, 'watched': True},
          {'name': 'Fight Club', 'year': 1999, 'watched': False}]
json.dump(movies, file_object)
file_object.close()

Once you run the code, you will see a file with the name movies.json which will contain the same output like the one above.

Working with your own types

We cannot simply store arbitrary Python objects as JSON. JSON can only deal with following objects:

  • numbers
  • character strings
  • None
  • booleans (True / False)
  • lists
  • dictionaries (only with character string keys)

Any object that is not one of the above must be converted.

import json

class Student(object):
    def __init__(self, roll_no, name, age):
        self.roll_no = roll_no
        self.name = name
        self.age = age

john = Student(1, 'John', 12)
json.dumps(john)

If you run the above code, it will fail and you’ll get a TypeError

To fix this, you have to define a default method for your class which should contain the following code:

def sdefault(obj):
    return obj.__dict__

Now you can encode the object into JSON with the following code:

json_data = json.dumps(john, default=sdefault)

Which will give the following output on printing json_data:

{"age": 12, "name": "John", "roll_no": 1}

You can name the default function anything you want.

REST API call with Basic Authentication in Powershell — September 14, 2016

REST API call with Basic Authentication in Powershell

Before we start looking at the code, let’s understand what Basic Authentication is all about.

Basic Authentication, in simple words, is a way of providing credentials (i.e. username and password) while making a request. See how it works in the diagram below:

Basic Authentication Process Flowchart

 

Now, let’s see how we can implement Basic Authentication using Powershell.

function Get-Data([string]$username, [string]$password, [string]$url) {

  # Step 1. Create a username:password pair
  $credPair = "$($username):$($password)"

  # Step 2. Encode the pair to Base64 string
  $encodedCredentials = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($credPair))

  # Step 3. Form the header and add the Authorization attribute to it
  $headers = @{ Authorization = "Basic $encodedCredentials" }

  # Step 4. Make the GET request
  $responseData = Invoke-WebRequest -Uri $url -Method Get -Headers $headers -UseBasicParsing

  return $responseData
}

You can use the above function in the following way:

$data = Get-Data -username user -password pass -url https://somedomain.com/posts
$dataToDict = $data | ConvertFrom-Json

In case, you’re getting the following error while calling the Get-Data method:


Invoke-WebRequest : The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.

Add the following line of code outside your function:

add-type @"
  using System.Net;
  using System.Security.Cryptography.X509Certificates;
  public class TrustAllCertsPolicy : ICertificatePolicy {
      public bool CheckValidationResult(ServicePoint srvPoint, X509Certificate certificate,
                                        WebRequest request, int certificateProblem) {
          return true;
      }
   }
"@

[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy

You can add additional attributes while forming the header like Content-Type, etc.