Saturday, April 30, 2016

CRMTest - Understanding the Github Repo


This video is created to walk you through the github project for CRMTest. In this video we took an example of a scenario - Interaction with public cloud. The intent was to show the starting point of understanding the whole flow. Hope you guys will like it. Please post your feedback if you have.

Tuesday, April 5, 2016

Salesforce Rendition + Microsoft Cognitive API = Happy Customers - Part 2

Yesterday I blogged about the idea of including microsoft's cognitive api to analyze image getting generated from Salesforce content rendition at http://nabaruns.blogspot.com/2016/04/salesforce-content-microsoft-cognitive.html.

However yesterday I was facing challenge in connecting the Salesforce file preview image to cognitive apis.

The major challenge was that Salesforce was not allowing me to get an insecure http image url for the preview renditions. Because of this I was not sure how the image url will be scanned by the cognitive apis.

But wait! there was another way to provide the content

Supported input methods: Raw image binary in the form of an application/octet stream or image URL.

The documentation said it can take raw image binary. This was a life saver for me as the rendition apis in Salesforce https://developer.salesforce.com/docs/atlas.en-us.chatterapi.meta/chatterapi/connect_resources_files_rendition.htm was capable of giving an InputStream for the generated thumbnail.

Now the only gap that need to be bridged was to convert this InputStream to InputStreamEntity and send it to the cognitive apis. Wallah! it worked seamlessly. :)

Here is the node.js client call that has been added after making the changes 

  1. Upload a File in Chatter feed
    response = crmtest.insertChatterFeedFile("/Users/Documents/pdf.pdf", "File Name","Title", accessToken);
    var fileId = crmtest.parseresponse(response,"id");
    • In order to upload a file in chatter insertchatterfeedwithfile method is invoked
    • First parameter is the file location
    • Second parameter will be the file name
    • Third will be the file title
    • Fourth will be the oauth accesstoken
    • Once the response is recieved it can be parsed through and corresponding file id of the uploaded file can be retrieved.
  2. Send the generated thumbnail preview from file rendition to Microsoft's cognitive api and get image analytics
    response = crmtest.analyzeimage(fileId, "<<Microsoft cognitive api key>>", accessToken);
    console.log(response+'\n');
    • Once file preview are generated, in order to get more analytics out of that send the fileId to the analyzeimage call with the Microsoft's cognitive api and accessToken
    • Raw Image is read from the chatter api and passed to the Microsoft's cognitive api. Note: Microsoft is not going to store the images as per its documentation
Execution time 
I uploaded the below seminar pic in my chatter feed 


Here is the output

-------------------------------------------------------------
-----------Analyze the generated file preview----------------
-------------------------------------------------------------
{"tags":[{"name":"indoor","confidence":0.98557054996490479},{"name":"auditorium","confidence":0.23898179829120636}],"description":{"tags":["indoor","room","large","building","sitting","table","auditorium","computer","people","man","group","laptop","crowd","standing"],"captions":[{"text":"a crowd of people in a room","confidence":0.26782138202681804}]},"requestId":"a23ecf3e-c0da-42b2-8d48-367848a97b93"} 

Note tags are nothing but the suggested topics in salesforce terms.

For more details refer to https://www.npmjs.com/package/crmtest which is updated with these changes. In order to contribute towards crmtest refer at https://github.com/Nabarun/CrmTest/

Hope this will help you in getting more out of Salesforce content.

Monday, April 4, 2016

Salesforce Content + Microsoft Cognitive API = Happy Customers :)

I cannot hold myself writing a blog on a most amazing video demonstration of cognitive api from Microsoft at https://www.youtube.com/watch?v=R2mC-NUAmMk. This video shows a blind person who is getting assisted with emotions through a cognitive api.

Wow!! first of all hats off to this thinking. I bet whoever is seeing this is having the same impression as I have.

They say innovation triggers innovation, and here it is. After watching this amazing video I went through the Microsoft cognitive api at https://www.microsoft.com/cognitive-services/

This to me is a goldmine to solve many real life problems.

The problem that I am focussed now is on driving topic suggestion out of generated thumbnail preview in chatter file post. The idea is as soon as the chatter post with file attachment is made, based on preview thumbnail generation, the salesforce user should be able to see the topics recommendation/suggestion.

Following step helped me to bring out a partial story around this. (I say partial as still I am trying to bridge the gap between salesforce rendition image url and microsoft's cognitive api.)

1. Signup to Microsoft cognitive api https://www.microsoft.com/cognitive-services/ Register for the free edition

2. After successful registration, click on "My Account" and make a note of key1

3. Follow the instructions at https://www.npmjs.com/package/crmtest and install crmtest by executing the following 
npm install crmtest

4. Observe crmtestclient.js and note the cognitive api calls in the bottom
//Suggest topics in preview image
console.log("-------------------------------------------------------------");
console.log("-----Suggest Topics in generated file preview----------------")
console.log("-------------------------------------------------------------");
response = crmtest.suggesttopicsinimage("<<image url>>", "<<cognitive api key>>");
console.log(response+'\n');

//Analyze preview image
console.log("-------------------------------------------------------------");
console.log("-----------Analyze the generated file preview----------------")
console.log("-------------------------------------------------------------");
response = crmtest.analyzeimage("<<image url>>", "<<cognitive api key>>");
console.log(response+'\n');

5. Note the first call is for getting the suggested topics from cognitive api, and the second call is for getting an analysis on the caption of the image .



First impression for a human eye will be what?

Execution time

This is a group of people standing in front of a building and watching an object. Sounds reasonable for me. When I executed the following script I was amazed to see the analysis.


var crmtest = require('./crmtest.js');

//Suggest topics in preview image
console.log("-------------------------------------------------------------");
console.log("-----Suggest Topics in generated file preview----------------")
console.log("-------------------------------------------------------------");
response = crmtest.suggesttopicsinimage("<<image url>>", "<<cognitive api key>>");
console.log(response+'\n');

//Analyze preview image
console.log("-------------------------------------------------------------");
console.log("-----------Analyze the generated file preview----------------")
console.log("-------------------------------------------------------------");
response = crmtest.analyzeimage("<<image url>>", "<<cognitive api key>>");
console.log(response+'\n');

Here is the output


-------------------------------------------------------------
-----Suggest Topics in generated file preview----------------
-------------------------------------------------------------
Topic Name: outdoor
Topic Name: road
Topic Name: building
Topic Name: street
Topic Name: front
Topic Name: large
Topic Name: man
Topic Name: riding
Topic Name: walking
Topic Name: people
Topic Name: sidewalk
Topic Name: city
Topic Name: standing
Topic Name: table
Topic Name: group
Topic Name: bus
Topic Name: train
Topic Name: holding
Topic Name: air
Topic Name: parked
Topic Name: board
Topic Name: track
Topic Name: trick
Topic Name: doing
Topic Name: white


-------------------------------------------------------------
-----------Analyze the generated file preview----------------
-------------------------------------------------------------
Category: building_ Confidence Value: 0.38671875
Category: others_ Confidence Value: 0.0078125
Category: outdoor_ Confidence Value: 0.01953125

Whoever eager to contribute to this framework, please note the github link for the develop branch is at https://github.com/Nabarun/CrmTest/tree/develop. Will really look forward for innovative enthusiast for contribution.

Life is not accumulation, it is about contribution.-Stephen Covey

Continued at http://nabaruns.blogspot.com/2016/04/salesforce-rendition-microsoft.html

Friday, August 7, 2015

Observer desgin pattern in Service Fabric Actor


First of all I want to thank Service Fabric Team to help me overcome and understand the actor logic with clarity.

In this blog I aim to create an Observer design Pattern of a real world sample using Service Fabric as a backbone. Apart from reliable service creation Service fabric also allows an author to create actor instances. Details are at here. These actors can be imagines as entities with specific business logic. These have proven to be fast and highly reliable in gaming solutions where response is the prime factor.

Today we are going to discuss of a widely used design pattern - Observer pattern in Service Fabric Actor to accomplish a threat analyzer for our test framework.

Problem Statement

Every company big or small emphasizes on testing frameworks. Usually these are typical dashboard based web solution which have compute role scheduling and triggering tests, monitoring them while they execute and storing the result in database. Test Dashboard shows a report view of the stored data.

We have often come across statements like

1.       How many tests are failing due to infra issue?

2.       How many tests are failing due to real code issue?

3.       What are the Azure Infra Issue type? etc

In my case we have a parser which goes and reads the database and do the bucketing based on different regex expressions. The challenge was the service was not reliable. This include service failures, connectivity issues etc. for our analyzer system, so we thought to try out Service Fabric and create a reliable service which will not only help us bucket the issue types but also will help us maintain the state for later analysis through dashboard. This was the time when we started going through service fabric actor model.

We heavily rely on Observer design pattern for our service. The intent was all the different issue types will be listening to a subject. The subject will at an interval of 30 min or so will query the database and push the list to all the observers. Based on issue type observers will add the object to itself.

When we tried to visualize this with Service Fabric, we had a picture which is described below:


In the above picture there is a stateless Fabric Actor which is our subject which will talk to the database at an interval of 30 min, and will push the objects to the observers. Observers will add the data based on its exception type.

Stateless Actor will push the data to the stateful actors, which will maintain the state as a list of issues. (Refer to the diagram below)

 

Later during analysis the user can directly call the actors and get their corresponding data. (Refer to the pic below)

 

Coding Time


 

Now let’s deep dive into how an actor works. As per my understanding author should keep these following things in mind. I will map the pointers to my example for better clarification

1.       Create a Serializable class which the Actor interface can call.

In this case my class was

[DataContract]

       public class ThreatState

       {

        [DataMember]

        public string ExceptionMessage;

 

        [DataMember]

        public string ExceptionType;

 

        [DataMember]

        public string TestRunId;

 

        [DataMember]

        public string TestId;

 

}

 

2.       Define a Subject Interface. Following OOD for implement to interface instead of concrete implementation, here is my ISubject interface

 

public interface ISubject : IActor

       {

        //Register Observer to Subject

        Task<bool> RegisterObserver(IIssueObserver observer);

 

        //Remove Observer from Subject

        Task<bool> RemoveObserver(IIssueObserver observer);

      

        //Set the Observable class state

        Task<bool> SetState(ThreatState state);

}

3.       Define a Observer Interface

 

public interface IIssueObserver: IActor

       {

        //Update the Observer object

        Task<bool> Update(ThreatState t);

}

 

Note: In both these interfaces we have implemented IActor which makes them actor interface. One primary thing to keep in mind is all the actor interfaces should return Task results.

 

4.       The Stateless Observable class will implement Actor and ISubject

public class ThreatObservable : Actor, ISubject

{…}

 

5.       For Stateful Observers here is the only change we need to make from Actor point of view

[ActorService(Name="AzureIssue")]

class AzureInfraIssueObserver : Actor<ThreatState>, IIssueObserver

{…}

 

[ActorService(Name="MachineIssue")]

class MachineIssueObserver : Actor<ThreatState>, IIssueObserver

{…}

 

[ActorService(Name="ProductIssue")]

       class ProductIssueObserver : Actor<ThreatState>, IIssueObserver

{…}

 

[ActorService(Name = "TestIssue")]

       class TestIssueObserver : Actor<ThreatState>, IIssueObserver

{…}

 

6.       Now in your Service Code during initiation add the different observers to the subject

 

public static void Main(string[] args)

        {

            try

            {

                using (FabricRuntime fabricRuntime = FabricRuntime.Create())

                {

                    fabricRuntime.RegisterActor(typeof(ThreatObservable));

                    fabricRuntime.RegisterActor(typeof(AzureInfraIssueObserver));

                    fabricRuntime.RegisterActor(typeof(MachineIssueObserver));

                    fabricRuntime.RegisterActor(typeof(ProductIssueObserver));

                    fabricRuntime.RegisterActor(typeof(TestIssueObserver));

 

                    var machineIssue = new MachineIssueObserver();

                    var testIssue = new TestIssueObserver();

                    var codeIssue = new ProductIssueObserver();

                    var azureIssue = new AzureInfraIssueObserver();

 

                    ThreatObservable observable = new ThreatObservable();

                    var registerMachine = observable.RegisterObserver(machineIssue);

                    var registerTest = observable.RegisterObserver(testIssue);

                    var registerCode = observable.RegisterObserver(codeIssue);

                    var registerAzure = observable.RegisterObserver(azureIssue);

 

                    if (!registerMachine.Result || !registerAzure.Result || !registerTest.Result || !registerCode.Result)

                    {

                        Trace.WriteLine("Registeration failed");

                    }

                    Thread.Sleep(Timeout.Infinite);

                }

            }

            catch (Exception e)

            {

                ActorEventSource.Current.ActorHostInitializationFailed(e);

                throw;

            }

 }

 

7.       This is what needs to be done from Service point of view.

8.       I tested the service with the following client

 

public static void Main(string[] args)

        {

            

            var proxy = ActorProxy.Create<ISubject>(new ActorId("ThreatObservable"),new Uri("fabric:/ThreatObserverApplication/SubjectActorService"));

            var lstThreats = GetListOfThreats();

 

            var result1 = proxy.SetState(lstThreats[0]).Result;

            var result2 = proxy.SetState(lstThreats[1]).Result;

            var result3 = proxy.SetState(lstThreats[2]);

            var result4 = proxy.SetState(lstThreats[3]);

 

            //Assert all the results to true and verify

        }

 

        private static List<ThreatState> GetListOfThreats()

        {

            var listThreats = new List<ThreatState>();

            listThreats.Add(new ThreatState() {ExceptionMessage = "Azure Storage Exception", ExceptionType = "Azure", TestRunId = "1234",TestId = "2345"});

            listThreats.Add(new ThreatState() { ExceptionMessage = "Bad Machine Exception", ExceptionType = "Machine", TestRunId = "1234", TestId = "2345" });

            listThreats.Add(new ThreatState() { ExceptionMessage = "Test Exception", ExceptionType = "Test", TestRunId = "1234", TestId = "2345" });

            listThreats.Add(new ThreatState() { ExceptionMessage = "Code Exception", ExceptionType = "Code", TestRunId = "1234", TestId = "2345" });

 

            return listThreats;

        }

 

That is it!!

As I wanted to keep this short, I wanted to focus on major interactions of your code with service fabric actor model. If you want to know more I have pushed a sample app at https://github.com/Nabarun/ServiceFabricSample.git. Let me know if there are any feedback or suggestion.