CNN: Understanding edge detection with an example

The convolution operation is one of the fundamental building blocks of a convolutional neural network and here we’ll discuss edge detection as an example to see how the convolution operation works. In my last post, we saw how the early layers of the neural network might detect edges and then some later layers might detect parts of objects and then even later layers may detect parts of complete objects like people’s faces.

Image Source: Andrew Ng Deep Learning Course

In this post, we’ll see how we can detect edges in an image.

Given a picture below:

Image Source: Andrew Ng Deep Learning Course

that for a computer to figure out what are the objects in this picture, the first thing you might do is maybe detect vertical edges in this image. For example, the first image on left has all vertical lines (marked in red), where the buildings are and pedestrians and so those get detected in this vertical edge detector output. And we also want to detect horizontal edges so for example, there is a very strong horizontal line where the railing is also got detected. Now let’s see in detail, how we detect edges in images like above?

Let us look with an example, Here is a 6×6 grayscale image and because this is a grayscale image, this is just a 6x6x1 matrix rather than 6x6x3 because they are on separate RGB channels. In order to detect edges or lets say vertical edges in this image, what you can do is construct a 3×3 matrix and in the terminology of convolutional neural networks, this is going to be called a filter and we’re going to construct a 3×3 filter or 3×3 matrix and now what we are going to do is take the 6×6 image and convolve it and the convolution operation is denoted by this asterisk(*) and convolve it with the 3 x3 filter. The output of this convolution operator will be a 4×4 matrix, which we can interpret as a 4×4 image. The way we compute this 4 x4 output is shown in fig below.

Image Source: Andrew Ng Deep Learning Course

Explanation: Looking on to the diagram to compute the first elements, the upper left element of the 4×4 matrix, what we are going to do is take the 3×3 filter and paste it on top of the 3×3 region of our original input image (left side) and what we should do is take the element-wise product (check the calculation method at the top of diagram in red). Similarly, we can calculate elements of a 4×4 image. These are really just matrices of various dimensions. But the matrix on the left is convenient to interpret as an image, and the one in the middle we interpret as a filter and the one on the right, we can interpret that as maybe another image And this turns out to be a vertical edge detector.

Let’s look at another example and see how exactly vertical edge detection is happening?. To illustrate this, we are going to use a simplified image. Here is a simple 6×6 image where the left half of the image is 10 and the right half is zero. If you plot this as a picture, it might look like this, where the left half, the 10s, give you brighter pixel intensive values and the right half gives you darker pixel intensive values. We are using a shade of gray to denote zeros, in this image, there is clearly a very strong vertical edge right down the middle of this image as it transitions from white to black or white to a darker color. When we convolve this with a 3×3 filter and so this 3×3 filter can be visualized as follows, where we can see that lighter, brighter pixels are on the left and zeroes in the middle and then darker on the right. What we get is the rightmost matrix.

Image Source: Andrew Ng Deep Learning Course

Now, if we plot rightmost matrix’s image it will look like that where there is this lighter region right in the middle and that corresponds to this having detected this vertical edge down the middle of our 6×6 image. In case the dimensions here seem a little bit wrong that the detected edge seems really thick, that’s only because we are working with very small images in this example. And if we use, say a 1000×1000 image rather than a 6×6 image then you find that this does a pretty good job, really detecting the vertical edges in our image. In this example, the bright region in the middle is just the output images way of saying that it looks like there is a strong vertical edge right down the middle of the image. So in this way, we detect vertical edges using convolutional operator.

In the next post, I will try to explain more edge detection examples also horizontal edge detection etc.

Happy learning!

WebDAV: an old horse but still useful!

My first encounter with WebDav protocol back in 2004-05 when I was writing the web part for SharePoint. WebDav is an old protocol of the ’90s but still very useful in certain scenarios.

WebDAV (RFC 4918) is an extension to HTTP, the protocol that web browsers and web servers use to communicate with each other. The WebDAV protocol enables a webserver to behave like a fileserver too, supporting collaborative authoring of web content. For an example, one can edit a word document online directly on the server using WebDav protocol. A web server that supports WebDAV simultaneously works like a fileserver. So, that’s a powerful capability.

In many of its use cases, WebDAV is being replaced by more modern mechanisms like Wikis, cloud solutions, etc. But it is still a reliable workhorse when the right servers and clients are matched, so it’s still encountered in many different applications.

Some of the servers which have implemented WebDav:

  • Apache HTTP Server
  • Microsoft IIS
  • Box.com
  • WordPress
  • Drupal
  • Microsoft Sharepoint
  • Subversion
  • Git
  • Microsoft Office
  • Apple iWork
  • Adobe Photoshop etc.

References:
WebDav RFC: https://tools.ietf.org/html/rfc4918

Reinforcement Learning Explained in brief for a layperson

As we know, Machine Learning algorithms can broadly be divided into 3 main categories:

  • Supervised Learning
  • Unsupervised Learning
  • Reinforcement Learning (RL)

Let’s understand in layman term what is reinforcement learning. The main thing RL does is Learning Control – This is neither supervised or unsupervised learning but typically these are problems where you are learning to control the behavior of a system.

Example:

How to cycle.
Remember the days when you are trying to ride a cycle…. It’s trial and error. Actually, it is some kind of feedback which is not fully unsupervised. So we can say that this is a type of learning where you are trying to control the system with trial and error and with minimum feedback. RL learns from the close interaction with the environment, close interaction means in this context is that an agent senses that state of the environment and takes the appropriate action. So the agent takes feedback from the close environment and we typically assume that the environment is stochastic means every time you take action you are not getting the same response from the env.

Apart from the feedback, there is an evaluation measure from the env which tells how well you are performing in a particular task. So each Reinforcement learning algorithm’s goal is to implement a policy that maximizes some measure of long term performance.

Just to summarize:

Reinforcement learning algorithm:

  • Learn from close interaction
  • Stochastic environment
  • Noisy delayed scalar evaluation
  • Learn policy – Maximize a measure of long term performance

Some applications:        

  • Game playing  – Games like backgammon (One of the oldest board game), Atari
  • Robot navigation
  • Helicopter pilot      
  • VLSI placement 

This was a brief introduction to RL for an easy understanding of the concept. For further study look for a good book or course.

My recommendation:

Happy learning!

elasticserach FScrawler intermediate file location? help

I am looking for a method that can intercept the output of FSCrwaler before this gets passed to the elastic engine for indexing.

Just take a case, what I have is a PDF file which I wanted to index plus I also have some attributes/metadata for the same PDF which are stored in the database. I want to index both the content (PDF file content + attributes stored in PostgreSQL) in a single index file so that I can refine my search criteria and get correct results. As far I understand for PDF indexing text needs to be extracted from the file and stored in a text format and then passed to the indexing engine. I am relatively new (an HTDIG veteran -:D ) to the elastic ecosystem so looking for a way to intercept the text extracted from a PDF file so that I can append other text (in this scenario fetched from PostgreSQL) and then pass to the indexing mechanism, a kind of single index file for both the content.

Wondering if anyone has encountered this kind of scenario and can provide some pointers? It will be good to know that where does FSCrawler stores intermediate files created during the indexing process in elastic search? Can we intercept them and add some custom info?

Comments will be much appreciated!

Thanks!

Resize a column in a PostgreSQL table without changing data: Repost

This morning I realized that one of the columns in the table I have recently created is running out of space. It is varchar(10) and I want to make it varchar(20) and of course wanted to do without losing any data which got filled in the last couple of days. Though there is a standard way to changing the size and type of the col using ALTER commands which can become tricky sometimes with the data. So I was looking for an alternate way and I found an awesome post on how to do this in an easy way without disturbing data, this is an almost 10 yrs old post so reposting the commands in case they get lost. For detailed reading please visit the blog link given in references.

Command to check the current size of a given column:

SELECT atttypmod FROM pg_attribute WHERE attrelid = 'TABLE1'::regclass AND attname = 'COL1';

In my case, the present size is 10 and I want to increase it to 25, so the command to update column will be as below:

UPDATE pg_attribute SET atttypmod = 25 WHERE attrelid = 'TABLE1'::regclass AND attname = 'COL1';

I hope this info and re-post helps.

Original blog link:
https://sniptools.com/databases/resize-a-column-in-a-postgresql-table-without-changing-data/

Deep learning on large images: challenges and CNN

Applying deep learning on large images is always a challenge but there a solution using convolutional but first, let’s understand in brief where is the challenge? So one of the challenges in computer vision is that inputs become very big as we increase image size.

Ok. Consider the example of a basic image classification problem e.g.; cat detection.

Let’s take an input 64×64 image of a cat and try to figure out if that is a cat or not? to do that we’ll need 64x64x3, (where 3 is the no of RGB channel) parameters, so the x input feature will have 12288 dimensions though this is not a very large number considering just the 64×64 image size which is very small there are lots of input features to deal with..

Say now if we take 1000×1000 image (1MB) which is decent in size but there will be 1000x1000x3 (where 3 are the RGB channels) ~ 3M input features

so if you put in a deep network then x will be 3M, and suppose if first hidden layer has 1000 hidden units then the total no of weights for a fully connected network will be (Weight matrix, X) ~ (1000, 3M) dimension means that the matrix will have 3 Billion parameters which are very very large, so with that much of data it is difficult to avoid overfitting in Neural Network and also the computational requirement to train the 3 Billion params is not feasible.

This is just for a 1MB image but in computer vision problem you don’t want to stick with using just tiny images using bigger images results in overfitting and huge input feature vector so here comes convolution operation which is the basic building block of Convolutional Neural Network (CNN).

Image Source: Andrew Ng Deep Learning Course



More on CNN in the next post.

Cheers!

Inspiration & source of understanding: Andrew Ng, Deep Learning Specialization

Prognostic Analytics for Predictive Maintenance, a case study

First, let’s try to understand the difference between prognostic analysis and predictive analysis. Predictive analysis tells that something is going to fail in future whereas Prognostic analysis tells that something is going to fail says in next some days/weeks/months. so there is always a time dimension factor in the prognostic analysis. Prognostic analysis can help in planning things in advance before the system actually fails results in saving resources and time. Let elaborate further with a case study.

Case Study: Let’s take a case where we want to use Prognostic Analytics for Predictive Maintenance in IoT based Systems in large plants e.g.; aviation, oil & gas, big manufacturer, etc. Running a prognostic model can help in finding out that performance of which controls system is degrading by analyzing the key sensor data of the past which can give an early sign that the system may go down and one can take precaution which can result in a big saving. The control systems which also include sensor infrastructure used in heavy industries generate tons-tons of data continuously and most of the time the data is decades-old, so basically companies stores the data but get never used. So there is a huge opportunity for heavy manufacturers to use the past data to get a good insight into the different parts of the system.

Technically, ML team can make a pipeline and stream the data which is coming out of the control system as it works and stream to a cloud-like AWS/Google/Azure or private cloud and then ML models can be run to check for the abnormalities and preventive maintenance can be planned. For an example, if you in a power plant and some crucial parts fails then someone from the supplier has to rush, take a plane and deliver and install which costs a lot of money but if we start doing the prognostic analysis we can get an early sign of which parts might fail and can procure.

Just to summarise, we can use the old data of the system/putting new data to the cloud and make preventive/prognostic analysis which can save money, resources and time.

References-Thanks: Dr. Harpreet Singh, I heard him on a podcast and highly impressed by his vision on data science and different use cases.

Debugging a crucial skill but very rarely taught

Listening on software engineering radio podcast Diomidis Spinellis mentions how debugging is so much important in software development but still, we don’t teach much in-depth this skill in our universities. I and believe any other programmer will agree that debugging tools are the key arsenal in fixing bugs and even understanding the system.

Either you use modern tools or just by basic print/printf statements that don’t matter. Students should learn these key skills and professors should emphasize on educating and not only in universities even in industry set-up when a new developer joins in there should be good exposure to debugging so that they dissect code base and become productive fast.

Worth considering I think …

What do you think? Please share in comments.

Outliers with Pankaj Mishra: podcast on entrepreneurship

Found this interesting podcast focusing on the Indian IT landscape, start-up, journey and entrepreneurship in general. The bio says ” A podcast about the ones who chose to take the road not taken often. It’s about the crazy and the curious. Those that dared to stand out, and stand-alone. It’s about their journey through hope and disillusionment, failures and pitfalls, joy and success, pain and bliss. It’s a candid exploration of experiences and ideas that have driven some of the shining stars, told as is.

https://soundcloud.com/factordaily

https://podcast.factordaily.com/

So far I have listened to a couple of them and found an enriching conversation.

Happy listening!

Date print formats Perl Date::Manip

Perl Date::Manip is one of the modules which I use a lot. It’s a wonderful lib and has very clean API with great documentation. Below is a quick look at Date::Manip print format options which sometimes is very handy. For detailed interpretation and other options encourage to go through Date::Manip on CPAN

Example:

my $present_date_hash = Date::Manip::Date->new("today");
my $present_date = $present_date_hash->printf("%Y-%m-%d %H:%M:%S");

Happy Coding!

also, can check my previous article on generating date patterns using Date::Manip

Books recommendation series: Essentialism: The Disciplined Pursuit of less

Book

Essentialism: The Disciplined Pursuit of less
Greg Mckeown
Essentialism – The Disciplined Pursuit of Less

Theme

The theme of the book is to find out what is truly essential? Focus on essential and eliminate everything else. Greg McKeown defines Essentialism as “Less but better”.

Takeaway

The key takeaway is that only once you give yourself the permission to stop trying to do it all, to stop saying yes to everyone, can you make your highest contribution towards the things that really matter. How often we say “yes” to everything, there is so much going on in our lives that nothing ever gets our true focus and attention. How can we give our best in any area when we’re being pulled in so many different directions? We can’t. We need to identify what is truly important, learn to say no to things that don’t fit into the “essential” category and simplify our lives. This book is an essential read for anyone who feels overcommitted, overloaded, or overworked.

Suggestions

Very good read. Highly recommended.

Note: My friend Rishi has done a detailed review of this book. Check this out here:

http://jigyasu.com/books/book-review-essentialism-by-greg-mckeown.html

Happy reading!

git push up to a certain commit

This is a quick share on git.
Scenario: I want to push my local changes to git but am having a few commits which I don’t want to push now. In other words, I just want to push changes till a certain commit.
Solution:
$git push <remotename><commit SHA>:<remote_branch_name>

To elaborate
First, fetch the SHA of the commit you want to push
$ git log

Copy the SHA and use the command below (Make sure that you replace the SHA of your commit with the given in the example).

$git push origin 7120f221660dad58d41b9ac729a22f08572b109:master

You are good to go. Your local commit of the given SHA is now pushed to server master keeping your local commits local only.

Please share in comments if you any other alternate way.

Thanks

Algorithms performance: big O notation: simplified short notes

 The big O notation is used to analyze runtime time complexity. big O notation provides an abstract measurement by which we can judge the performance of algorithms without using mathematical proofs. Some of the most common big O notations are:

  • O(1) : constant: the operation doesn’t depend on the size of its input, e.g. adding a node to the tail of a linked list where we always maintain a pointer to the tail node.
  • O(n): linear: the run time complexity is proportionate to the size of n.
  • O(log n): logarithmic: normally associated with algorithms that break the problem into similar chunks per each invocation, e.g. searching a binary search tree.
  • O(n log n): just n log n: usually associated with an algorithm that breaks the problem into smaller chunks per each invocation, and then takes the results of these smaller chunks and stitches them back together, e.g, quicksort.
  • O(n2): quadratic: e.g. bubble sort.
  • O(n3): cubic: very rare
  • O(2n): exponential: incredibly rare.

Brief explanation:     
Cubic and exponential algorithms should only ever be used for very small problems (if ever!); avoid them if feasibly possible. If you encounter them then this is really a signal for you to review the design of your algorithm always look for algorithm optimization particularly loops and recursive calls. 

The biggest asset that big O notation gives us is that it allows us to essentially discard things like hardware means if you have two sorting algorithms, one with a quadric run time and the other with a logarithmic run time then logarithmic algorithm will always be faster than the quadratic one when the data set becomes suitably large. This applies even if the former is ran on a machine that is far faster than the latter, Why?

Because big O notation isolates a key factor in algorithm analysis: growth. An algorithm with quadratic run time grows faster than one with logarithmic run time.

Note: The above notes are for quick reference. Understanding algorithmic performance is a complex but interesting field. I would recommend picking a good book to understand the nitty-gritty of big O and other notations.

Python learning: scrapbook

While browsing my Evernote I found a scrapbook which I have made while learning Python some years back. Thought to share if this helps someone. I am pasting directly (no editing so there might be some spell and grammar mistake).

  • Python 2 division, just use integer part (3/2=1) whereas Python 3 uses real division 3/2 = 1.5
  • Strings in Python are immutable means you can’t change the in-place value of a char. Once string is created you can’t change/replace its elements
  •  s= “Hello World” s[::-1] this will reverse string s “dlroW olleH” double colon is used to tell the range and also how many elements can be skipped
  • if you want to use Python 3 functions in Python 2 then use ‘from __future__ import print_function‘ and similarly other functions 
  • List are mutable but tuples are not mutable (does not support item assignment) aka immutable, fewer methods in tuples then why to use instead of a list? The key is immutability. in a program if you want sequence/Val does not to get changed then tuple is a solution e.g.; storing calendar dates which know will not change during your programs. 
  • Set is a collection of un-ordered unique items it looks like a dictionary (in notation) but only keys which are unique. It can help in removing repeated items means you can use set to cast list.
  • List comprehensive are an excellent way to write clean and efficient code – they are actually de-constructed for loop flatted out in a list
  • Lambda expressions can be used to shorten function this is really useful when used with map(), reduce() and filter() functions
  • First class functions: Treat functions like any other object, we can pass functions, we can return functions, we can assign functions to a variable
  • Closure: Closure takes advantage of first-class functions and returns inner functions and variables local to them.
  • Decorators: It is a function which takes another function as an argument and returns as a function without changing the source code of the original function. Decorator allows easily to add functionality inside our wrapper without modifying original function. 

Note: These are notes for quick reference. If you are serious in learning Python I encourage you to take a book or a tutorial.

Enjoy learning! More to come …

Microsoft Visio and infamous Windings font

Another Microsoft misery 😢 😢

This time, I am struggling to handle the bullets in SVG output of my drawing.  What I found is that  MS Visio still uses windings/symbol fonts to make the bullets. Don’t know why? When are they going to change this and start using standard Unicode chars? Can anyone from MS please explains to me why they don’t believe in standardization?

IMHO Please use Unicode everywhere be it HTML generated from Word or Visio etc. 

Thanks for reading!

IE11 issue with “arrow” and “includes” Javascript method and an alternate solution

Again one more Microsoft problem 😢 😢

Yesterday, I  suddenly observed that one part of my newly developed feature was not working in IE11. This is a simple feature which contains two select boxes, user can select the one or multiple items from the first select box and copy to another select box. I have added a kind of logic there which checks before copying if the item(s) to be copied are already in list 2, if true, alert the user and skip copying duplicates. While developing I checked this feature in chrome, Edge and Firefox and all work perfectly!  I couldn’t check in IE11 and that was my mistake as IE is infamous for such issues. While debugging I saw that IE11 was crashing on the places where I have used the functions  => and .include. Actually, the arrow function is not supported in IE 11. You can refer to this compatibility table: https://kangax.github.io/compat-table/es6/ to get an overview of what is supported where and to what extent in a detailed fashion. You may read more on arrow functions at below link: 

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Arrow_functions

Check below a small demo which shows the problem and an alternate which solves the IE11 issue.

Demo:

**********************************

JSFiddle (Fails in IE11)

JSFiddle (Works in IE11)

**********************************

Github:

code (Fails in IE11) 

code (Works in IE11) 

 

**********************************

Happy coding!

Perl development journey and books collection

While doing routine cleaning of my personal library I was surprised to see the Perl book collection I have made over the period of time. My Perl dev journey started in a full-fledged manner way back in fall 2007. Prior to that was mainly developing using C, C++, assembly language. My first impression with Perl was not very exciting mainly due to ugly syntax and the way the OO is achieved and being from C++ background initially it was really difficult to grasp. But over the years working with language and while developing a large scale web application I learned a lot of nitty-gritty of the language and still learning…  Today I can vouch for Perl for its speed, portability, great module system CPAN and excellent dedicated community. Thanks to all the module authors and contributors on Perl Monks and StackOverflow. You guys are amazing! 
Now, the books which helped me immensely to wrote better Perl programs.