Blog: My thoughts Joyous

Programming keep me busy

Time flies. I though have an idea that I didn’t blog for long time, but really got myself surprise that it is over three months now. Was busy programming and finish couple of long delayed projects and life is taking new shape. In recent past, I developed new methodology for my programming and that process is still going on. I am excited for couple of things that will happen to me sooner some personnel things, some technical upgrades. In last couple of month I developed my new framework of working in PHP projects. They now accelerated my programming by at least double speed and at time even 4 times to my old speed. This give me enough time to enchance security and speed of overall software. Software that I develop are more scale-able in natural manner than before. 

But that is what I did in PHP, and now the time has come to upgrade my .NET skills, with my recent projects need new things in .NET and Windows Azure it looks so exciting next couple of month. Things involve need Parallel programming, Azure hosting, upgrading 3 yrs old PHP project to latest methodology and programming environment. 

I need 3-4 months more for me to understand more in-depth of programming speed. The software I develop are majorly the same just they are more mature and more self reporting. 

Hope I can keep up the work pace better than before and deliver much more in much less time. I will post my framework sometime in future when I upgrade it to public share able level. But not quite ready for it.

Article Blog: My thoughts Database MS SQL Server MySQL SQL SQlite

Database Optimization

Well, most of program use database for their web application. Eventually all web application does have database. But how many of those web application really works good. Well quite few. I am in web programming for more than 7 years now and during these years I have seen lot of developers building website using PHP or ASP.NET or ASP or PERL using SQL SERVER, MYSQL, Oracle or other database, but I always have a sense of making best application than them. But today I when I look back I realise that I was bad at writing code and making application in my initial years. Not that I don’t want to write a good code, but the fact my approach was not right. Some of the important thing that I learn over the year based on my experience are listed below. However before that I want to tell that not just me but all of us have read them in books but most of us might not have used it thinking we don’t need it as things works otherwise as well. So major stuff are:

1. Fast Database

2. Object oriented Code

3. Serialize flow of application.

In this post I try to mention Only Fast database. Well with Fast database I do not mean to pick the Fastest database on planet through bookmark testing, nor does I mean to use heavy hardware to make database run faster. These two things can really make your database appears to be faster but not really fast. I often ask a question in forum that I have few million records to parse, and can the DB engine handle it. I always get one answer if a database cannot handle few million record why we really need it.

This was very true statement, but do you really facing problem when you have only half million record and your site start taking 20-30 second to access database ? If so, then welcome to my past world. Well don’t worry you don’t need new hardware, you need new approach of programming your software and most importantly you need better database structure.

Starting with most commonly used stuff and I guess most importantly forgotten stuff.. Indexes. Yes, this is most common solution to make your site fast. Okay, so you already have define primary key in your table and it auto create index for it, but still not good enough. Well you need to index columns that you have used in your search and or in Join. Making index of one column does help, but making index of combination of 2 columns make it much better in queries that use the search on those two fields. For example, if you are searching Employee Last Name and their City from Employee table, having index on LastName and City seperately produce result slower than having index on Lastname and city name.

Now moving to next part of it. I have a sql query that has Client Code and User code in seperate column and I have a text box that allow user to enter ClientCode – UserCode in there, and I have to search it in database. I got little lazy and what I did I took the Textbox value as it is and did something like

where ClientCode + "-" + UserCode = @MyClientCode

Now, this looks good and works well for me, but this is not a good approach of writing it. In where clause never do the combination of column like that. Search easy column seperately, i.e.

where clientCode =@ClientCode and UserCode =@UserCode

the difference make your search works 10-15 times faster if you tend to have lot of records.

Third thing that comes to my mind is normalization of database and then creating views with indexes on view to join them if needed. I often see that database is not normalized and lot of Text field are used to store combination of values. For example if you want to have a product in different category then make a table Product_category, which store product ID and CategoryID, instead of storing comma seperate category IDs in product table. This seems awkward at first to change, but advantages are real. You can search product on category much faster as only Integer values are compare against the Regular expression match in comma seperated values. You can index the value in relation table, but not in text field. This approach is very common in developer in sub continent. If I am not wrong 70% of sites I see from other developer has this mistake.

Above are some error in design of database I have seen and some of them I did at my start of career, but I change my approach and today I am handling database with more than 10-20 million entries easily. Infact my site took less than 1 sec to search with a three table join. I remember the old version of this join that took 15 secs. I did great optimization there.

Anyways, making a software is your work, making it great is your choice.

Blog: My thoughts Rants

Internet: Fun or frustration.

Since the interception of Internet, human being is evolved as more social animal then before. With use of internet increasing among human it was become fairly easy to stay in touch with your loved one. However as soon as Internet crazy is about to reach its height, mobile phone aka smartphone really turn the table in further fuel addition to social human being. However with Internet becoming an essential part of human life, getting a quality service is another tough call for them. It frustrate me most when my Internet is not working as compare to having a dead phone line. Need of quality service such as AT&T, O2, Verizon and other international player that provide quality Internet in mobile device are still very rare. It is even worst in country like India, where you expect service to be even better than those in most part of world.

With only less than 10% indian population is using Internet on daily basis, India still stand on top 5 internet using country. I have use quite a few internet service in recent past, but no real taker of quality of service. Even the most expensive service is just satisfactory and not really good. When I got my first smartphone, my excitement of using Internet on the go is vanished within 24 hrs, a 3G device was running less than 1MB/s speed having a 3G data plan on it.

India government has define some minimum speed limits, though those limits are way less than global standard and when it comes to quality of service they are even worst. Though, if I take a  look from what we have 5 years ago, it looks bit satisfactory. I remember I use Dial up connection as my first internet, and today I am using 2MB/s Speed on my desktop and 3G on my mobile device. Speed is good so as need of having speed. It took me 7 days to download a JDK in 1999, today it take me less 7 minutes to do so. It would be good, if quality of internet improve a little further, though I get 99% continued connectivity with my broadband connection, but having a consistent speed is big issue. Failing DNS is second major concern. Hope to get a better quality in next few months with evolution of need and standard supply/demand format of how economy works. If they provide speed/quality they will get demand and hence their supply increase their demand.

Article Blog: My thoughts

Are you Marketing or spamming?

Today I delete bulk of comment on blog moderation list. These guys took some effort to put their comment and I delete it. Because they are not commenting on article or entry, they are simply marketing their stuff. There is often such comment you will find on your websites, but is this marketting really useful to post such comments to personnel blog or company websites.

Hello, I am Mr. X and can help you get top rank on Google Search result. Try our SEO service.

I got these message very often, but hey I am already on top 10 results for keywords I want. Did you ever research that ? More over, this is not a business site, so why I want anyone to find my blog. I write it for my pleasure of sharing. If someone need my blog they always find it anyways.

If you are really wanting to market your service, there are certain rules to follow:

Blog: My thoughts

HostGator: Impressed me, but still not good

After a very bad experience in hosting service and try like of godaddy, 1and1, bluehost and few other, I finally reach HostGator. They are still a traditional hosting provider but at least their server works. It is not that you cannot find a good hosting then hostGator, but comparing to $$ spend on hosting decide the quality a lot. I use company like Rackspace and, they are really what a quality hosting should be, but considering their fee, it is not reachable to every day to day client. 

However, I use their USA server and it was not quite good, maybe too much overloaded, though for my blog I use their Indian Server and since it is new server and probably not so much loaded yet, it works great for me from last 3 months. They have down time of about 10-15 minute here and there, maybe server just get too overloaded at time, been a shared hosting server, but otherwise it is normally a good speed. My clients are happy that my project management system finally works for them, and this is great achievement in itself.

Anyways, my new working environment is going solid. Increase my productivity a lot just waiting for good results to come with time.

Blog: My thoughts Rants

New Future: undecided

Been involve in technical field, I have encounter lot of technical hurdles. There has been some new problem always, and I somehow manage to escape them by solving those problem. Though not all problems are solvable is what I learn overtime and you need to take different route to nullify the effect of such things. In last 3 years, I had a good time with all freedom I enjoying working on project that I love to and try to finish and accomplish them with my methods. It has been a great evolution process for my knowledge.

Though it doesn’t seems as comfortable now, as the difference between me and my dream has grown bigger. My dream is around building a team of technical person help or work with me to achieve goals we set. We want to achieve the standards we set over other counterpart of ours. Though the term team seems to be the biggest challenge to get at this stage. With economy going no where, finding good person is becoming equally hard. About 18 months ago, you will find all sort of talent around you, but now I see a downside which is effective by two things mainly:

1. No one is willing to take responsibility of work. It is not that this is uncommon thing to human, but it has increased a level where you just cannot trust your employee to any degree. Even if you pay them double the salary, but they know they are just employee and one day they use your double salary to get hired in another company with even more money.

2. Freedom from work, eventually you will read on internet or articles in magazine that people look for freedom at work, but I said they look for freedom from work. If you don’t ask them to follow your well tested methodology and don’t ask them to finish in time, they are happy to work for you, but otherwise they says we need freedom to work our way and hence they don’t like you.

These unfriendly act of candidate here leave me in two mind to go with them or not to go with them. I see some of them does has good thinking brain, but they are not willing to work. I can make good software but I am not god to make good human.

Always in search of good developer who help me go along, but I decide not to work with one who is not more than an employee. I will now work for great peoples only, either they are my client or my team members.

Article Blog: My thoughts Operating System

Why we don’t need VPS?

Late last year, I was pretty happy to have a budget to take a VPS for my company. This was first server that we own. What a great feeling to have it. But soon the great feeling becomes a nightmare. It was not that hosting company is not that good, it was just that the VPS are not good in general. I am talking about it after using atleast 10 VPS server from 3-4 different companies. Fortunately I invest only in one of them.

Coming back to topic, here are reasons why I hate VPS now:
1. VPS is said to be cheaper, But for whom? You or hosting companies? Companies with good reputation or resources gives a VPS starting at $60 per month for just a 1GB RAM, with 30GB HDD. IF you compare it with Dedicate server with low end they ends up with 160 GB HDD and 2 GB RAM. for about $100. But to get that configuration you need to spend over $100. So comparatively they are not cheap.

2. When I buy a VPS I read that my VPS will be hosted on Cloud network and hence if a VPS fails for some reason it get up again on next node in network and reduce the downtime. Unfortunately my VPS Hosting GRID failed and it dump my data for 48 hrs, before they can fix the node. I was not alone but all people having their VPS on that grid lost valuble 48hrs waiting for their server. Later it took 15 days to get my hosting to move my VPS to new GRID as even after 48 hrs they said they still need to do few maintenance and my VPS will do roller coaster during that time. If I have Dedicated server and if some hardware fails it took less than 2 hrs to get it up live. But if VPS grid’s hardware fails who grid gets overloaded from other VPS and reduce performance. Plus changinge their hardware wasn’t easy. This is what I had experience on my server and unfortunately on client’s hosting when their Hosting [different my mine] fails for them too.

3. VPS should not be confused with Cloud Hosting. VPS are virtual machine, and even if they are hosted on Cloud Network they are not Cloud hosting for your purpose, they are as good as dedicate machine. IF your VPS fails, or shutdown, your website will be down. You won’t get shifted or reboot, becuase machine is down not a network. It is like notepad crashing on your computer. It do not effect computer otherwise. Cloud hosting are like shared hosting, where multiple computers store data, and a data replication happens on fallback node. then your webserver runs on different hardware on same grid and other application as well. So if say one web server fails there are other webserver available to take load of your application. But this is note the case with VPS machines. IF you take VPS you take a 1990’s computer as server don’t expect too much.

4. VPS is a concept of running more than one Operating system on given hardware. Like running Parallel to run Widows on Mac. Though it is been said that they share resources, but actually if you have a computer with 4 GB RAM running 2 VM with 2 GB RAM. do you think you get full 2GB RAM performance? the answer is NO. because at 4GB RAM there are 3 OS running. and one is using 4GB, while other 2 are using 2GB each, so they need 8GB in total, but it is not possible because we have 4GB. So the host operating system scarifice some of his RAM, maybe 3GB but it still be a 5GB. Same happens with CPU processing, Same with HDD. A HDD writes a 1xX speed if it has to write it for 3 OS will it write faster? no it still write 1xX and each os get only 1/3xX of speed.

In nutshell, if you are trying to save money by using VPS, then you better reconsider using a Shared hosting or a Dedicated server. Hosting 10 site on VPS will cost you $6 per month with no performance, where as shared hosting from decently good host will cost your just about same, but much better performance. And if you have enough money why not a Dedicated server or a Cloud hosting like Microsoft Azure, or Amazon S3 like services ?