Technical Debt

This article has got published at -
http://www.mbaskool.com/business-articles/operations/7910-technical-debt-causes-and-solutions.html

History –

Ward Cunningham, a software engineer from America coined this terminology in 1992. He is one of the pioneers of programming methodology like extreme programming. With his several years of experience and innovative ideas, he has been adding new terms to software dictionary.

What is Technical Debt?

Lets us first recall the meaning of the term Debt. As we know very well, debt word is used in financial domain frequently which means a kind of monetary libality of a party or individual.
Debt is a sort of bond on somebody for accomplishing the agreed contract work.

The same concept gets applied to technology field. Widely used in software development field, technical debt talks about the tasks not accomplished which were supposed to be part fo the entire project work.
A system can be in the state of completion with technical debt but definetely contains hidden deficiencies which mars the quality.

In the beginning, the pending activities do not stop the servival or continuation of the project and do not appear fatal to project life cycle. But these works do add values to the project deliverables and the development work.

Moving ahead, the heap of the pending tasks gets large and starts impacting the quality of the work badly.
Lets us have an example to get it understood –

Consider the case of developing a software product handling life cycle of capital market financial product like equity & derivatives.
The well progressing development has some flaws where some sleeping partners are not adding the test cases as part of automated testing. The missing test cases are compensated by the manual testing work.
The tight scheduled project with lack of code review has allowed this to creep in and this pending tasks has started technical debt pool.

Though in the beginning it is not appearing as load, for the subsequent work in the same area will require repeated manual testing. So the debt will be punishing the developer to carry out manual testing every time and make the testing work inefficient.
Also, this will be forever and will make the person its pray who is working on the piece of code.

One can imagine the scene where new functionality are being added to the existing system and will require manual regression testing because of missing test cases or the technical debt.

Prominent Causes –

1. Individual Poor performance-
Poor performance from a technical person will allow initiating such deficiency in the system. Individual level low quality performance is one of the prominent causes of this debt which gets added upon as the system grows.

2. Inefficient Quality & Control Mechanism -

Lack of good review techniques and implementation of the technical works accomplished are another strong reason of such debt creation. Review should be accompanied with appropriate changes and feedback.

3. Carry forward of the debt -

Usually the new person carries forward the technical debt created by the earlier person on the same work.
Many times, in big systems it becomes difficult to get rid of such earlier backlogs and the debt appears as permanent hindrance.

4. Nature of the Biz need -

Tight scheduled biz requirements many times do not provide scope for tracking the technical debts and thus
compromise with the quality of the work. This factor is something out of the control of the technical team.

5. Other factors -

There could be other factors as well like - poor management support, lack of quality standards, inefficient organization technical culture, lack of leadership etc.

Some of the remedies –

Few steps to overcome this scenario are -

1. Adhere to good technical standards and quality practices.
2. Adopt practices for reducing the technical debt with the ongoing works.
3. Developing the environment for individuals to take proactive steps for avoidance of initiating technical debt.
4. Training and awareness for such aspects
Lets us enjoy our technical work without having future debts!

Python Threading Features

Threading

What is threading?
Light weight process

Benefits
Shared resources
Switching of threads
Efficient utilization of CPU etc.

Python Module
thread

Contents of the module ‘thread’ -
['LockType', '__doc__', '__name__', '__package__', '_count', '_local', 'allocate', 'allocate_lock', 'error',
'exit', 'exit_thread', 'get_ident', 'interrupt_main', 'stack_size', 'start_new', 'start_new_thread']
e.g. -

import thread
import time

# thread function

def print_time( threadName, delay):
count = 0
while count < 5:
time.sleep(delay)
count += 1
print "%s: %s" % ( threadName, time.ctime(time.time()) )
# Create threadds
try:

thread.start_new_thread( print_time, ("Thread-1", 2, ) )
thread.start_new_thread( print_time, ("Thread-2", 4, ) )
except:
print "Error: unable to start thread"
O/p -
Thread-1: Thu Aug 06 15:42:17 2013
Thread-1: Tue Aug 06 15:42:19 2013
Thread-2: Tue Aug 06 15:42:19 2013
Thread-1: Tue Aug 06 15:42:21 2013
Thread-2: Tue Aug 06 15:42:23 2013
Thread-1: Tue Aug 06 15:42:23 2013
Thread-1: Tue Aug 06 15:42:25 2013
Thread-2: Tue Aug 06 15:42:27 2013
Thread-2: Tue Aug 06 15:42:31 2013
Thread-2: Tue Aug 06 15:42:35 2013

Python Module
threading ( Part of Python 2.4 )

Content of Threading Module
['BoundedSemaphore', 'Condition', 'Event', 'Lock', 'RLock', 'Semaphore', 'Thread', 'ThreadError', 'Timer',
'_BoundedSemaphore', '_Condition', '_DummyThread', '_Event', '_MainThread', '_RLock',
'_Semaphore', '_Timer', '_VERBOSE', '_Verbose', '__all__', '__builtins__', '__doc__', '__file__', '__name__',
'__package__', '_active', '_active_limbo_lock', '_after_fork', '_allocate_lock', '_counter', '_enumerate', '_format_exc', '_get_ident', '_limbo', '_newname', '_pickSomeNonDaemonThread',
'_profile_hook', '_shutdown', '_sleep', '_start_new_thread', '_sys', '_test', '_time', '_trace_hook', 'activeCount', 'active_count', 'currentThread', 'current_thread', 'enumerate', 'local',
'setprofile', 'settrace', 'stack_size', 'warnings']

Content of Threading.Thread class
_Thread__bootstrap', '_Thread__bootstrap_inner', '_Thread__delete', '_Thread__exc_clear', '_Thread__exc_info',
'_Thread__initialized', '_Thread__stop', '__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__',
'__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_block', '_note', '_reset_internal_locks', '_set_daemon', '_set_ident', 'daemon', 'getName', 'ident', 'isAlive', 'isDaemon',
'is_alive', 'join', 'name', 'run', 'setDaemon', 'setName', 'start'
e.g. –

import threading
import time
exitFlag = 0
# inherit from threading.Thread class
class AvaneeshThread (threading.Thread):
def __init__(self, threadID, name, counter):
threading.Thread.__init__(self)
self.threadID = threadID
self.name = name
self.counter = counter

# override the run() method
def run(self):
print "Starting " + self.name
print_time(self.name, self.counter, 5)
print "Exiting " + self.name

# thread functionality method
def print_time(threadName, delay, counter):
while counter:
if exitFlag:
thread.exit()
time.sleep(delay)
print "%s: %s" % (threadName, time.ctime(time.time()))
counter -= 1

# Create new threads
thread1 = AvaneeshThread (1, "Thread-1", 1)
thread2 = AvaneeshThread (2, "Thread-2", 2)
# Start new Threads
thread1.start()
thread2.start()
print "Exiting Main Thread"

o/p -
Starting Thread-1
Starting Thread-2
Exiting Main Thread
Thread-1: Thu Mar 21 09:10:03 2013
Thread-1: Thu Mar 21 09:10:04 2013
Thread-2: Thu Mar 21 09:10:04 2013
Thread-1: Thu Mar 21 09:10:05 2013
Thread-1: Thu Mar 21 09:10:06 2013
Thread-2: Thu Mar 21 09:10:06 2013
Thread-1: Thu Mar 21 09:10:07 2013
Exiting Thread-1
Thread-2: Thu Mar 21 09:10:08 2013
Thread-2: Thu Mar 21 09:10:10 2013
Thread-2: Thu Mar 21 09:10:12 2013
Exiting Thread-2
Synchronization –
Need?
Available objects are - 'Event', 'Lock', 'RLock', 'Semaphore'

1. LOCK -
import threading
import time
class AvaneeshThread(threading.Thread):
def __init__(self, threadID, name, counter):
threading.Thread.__init__(self)
self.threadID = threadID
self.name = name
self.counter = counter
def run(self):
print "Starting " + self.name
# Get lock to synchronize threads
threadLock.acquire()
print_time(self.name, self.counter, 3)
# Free lock to release next thread
threadLock.release()
def print_time(threadName, delay, counter):
while counter:
time.sleep(delay)
print "%s: %s" % (threadName, time.ctime(time.time()))
counter -= 1

threadLock = threading.Lock()

…
2. Semaphore –
Defined number of thread with accecc control mechanism.
maxconnections = 5
...
pool_sema = Semaphore(value=maxconnections)
pool_sema.acquire();

conn = connectdb();
... use connection ...
conn.close();
pool_sema.release();

3. RLock –
This is a special kind of Lock where a thread can utilise the Lock multiple times. So,
more than once acquire() and release() are called on the Lock.
This is used when recursive Lock.acquire() and Lock.Release() are called by a thread.
lock = threading.Lock()
lock.acquire()
lock.acquire() # the second acquire() will fail
lock = threading.RLock()
lock.acquire()
lock.acquire() # the second acquire() will not fail

4. Condition –
It denotes some state change and access of a thread can be controlled for
synchronisation.
# get the condition object
condition = threading.Condition()

#acquire the condition obj
condition.acquire()
# signal for any state change
condition.notify() # signal that a new item is available

# release the condition obj
condition.release()
Thread Pool / Queue -
Get a suitable datastructure like dict to hold worker threads created in advance for
any coming tasks.
e.g. – ATM applicaton
A priority system can be implemented and used with the thread pool functionalities.
e.g. –
Having priority field in customised thread class.
class AvaneeshThread(threading.Thread):
def __init__(self, threadID, name, counter, priority):
threading.Thread.__init__(self)
self.threadID = threadID
self.name = name
self.counter = counter
self.priority = priority

Python class sample code

Python is object oriented language with support to many OO concepts.
A very simple sample of class and member methods -

class MyClass:
def __init__(self, id):
self.id = id;

def showId(self):
print "Value of Id is :"+str(self.id);
return;

def __del__(self):
print "Deleting the object...";

myCls = MyClass(10);
myCls.showId();
del myCls;

O/P -

Value of Id is :10
Deleting the object...

02-08-2013

Few Oracle SQL Performance Improvement Tips

Some of the performance improvement criteria for Oracle SQLS are below :

1. IN and NOT IN Operators -
These operators are slower than the operators EXISTS and NOT EXISTS.
Use EXISTS and NOT EXISTS with the target select statement.
This also gives chance for getting the index (if any) in the targetted sql utilised.
e.g. -
SELECT
BK.fm_book_id IN (select fm_book_id from fm_book_group_link where fm_group_id=128)
FROM BOOK BK

can be replace by

SELECT
BK.fm_book_id EXISTS (select fm_book_id from fm_book_group_link where fm_group_id=128)
FROM BOOK BK

2. LIKE operator -
Use leading character for pattern matching like 'A%' instead of '%A'.
Also, use TEXT field INDEX created and use CONTAINS in place of LIKE.
The entire story is at - http://www.orafaq.com/node/1918

3. Take care of system functions -
Try to avoid using system function as they can add up to the performance of the processing.

4. Ordering of the conditions in WHERE clause -
Ordering of the conditional statements in WHERE clause affects the table scan effort and impacts performance.
e.g. -
Use the statements having number (constant) comparison first, as this will filter the number of records and reduce the table scan.

Hope this will benefit somewhere :) Good Luck !

Avaneesh S. Tiwari
12-07-2013

Operator overloading

C++ supports Polymorphism concept with giving different & sepcial behaviour depending upon the context. Generally, this polymorphic behaviour is at compile time or at runtime.
Function overloading & operator overloading are two compile time polymorphism examples.
For this article, we will get an introduction to Operator overloading.

Opearator Overloading -

The existing supported operators are extended for their functionalities ( behaviour ) by operator overloading. This supports the extensibility concept of C++ which has made it to produce great amount of customised libraries.
Operator overloading adds up to the functionalities of the operator with preserving other features like presedence, number of arguments to be operated & basic functionality etc.


Prototype -

return_type operator op (argument list )
{
...
}
op - the operator to be loaded

The keyword "operator" is used for denoting the function as operator overloading function.
The function can not be static. But it can be made friend function.

Limitations -

There are operators which can not be supported for overloading as an exception -
a. Scope resolution operator (::)
b. size of operator (sizeof())
c. conditional operator (?:)
d. pointer to member variable (.*)

Also, when we make the function as friedn function, it can not overload the following operators -
a. =
b. ()
c. []
d. ->


Usability -
Its used for extensibility, reusability with additional features.

Example -

// code of Test2.hpp file

#include
#include

using namespace std;

template
class MyList : public std::list
{

public:

// Default ctor
MyList ( ) : std::list ( ) { };
MyList (size_t n) : std::list (n) { };
MyList (size_t n, const T& val) : std::list (n, val) { };

// overloading operator "[]" as member function
// returns the ith element from the list
const T& operator [] ( size_t i) const
{
typename std::list::const_iterator c_it=this->begin ( );
typename std::list::const_iterator cv_end=this->end ( );

size_t intCnt = 0;

// loop till ith element
while ( intCnt != i && c_it != cv_end )
{
c_it++ ;
intCnt++;
}

// return the ith element value
return *c_it;
}
};


// code of Test2.cpp file


#include "stdafx.h"
#include "Test2.h"

int main(int argc, char* argv[])
{
// instantiate the class MyList for string type values
MyList obj;
//add values
obj.push_back("Avaneesh");
obj.push_back("Anand");
obj.push_back("Amar");

//get the first element
std::cout< //get the second element
std::cout< //get the third element
std::cout<
return 0;
}

Pass by Reference in C++

Always try to pass objects by reference in C++ in funcation calls as it offers following advantages -

1. Efficient -
Pass by refernce just uses aliases for the object being passed. It does not make copy of the object.

2. Safe -
As reference variable is used for passing the object, possibility of NULL value is avoided.

3. Multiple value return -
Using the reference variable, one can make changes to the members of the object and return as a reference object. Thus, many changed values can be returned.

Embedding SQL into Unix Script

Embedding of SQL query in Unix Script -


The automation usability of the Unix Scripting can be enhanced with embedding the SQL queries into a Unix script file.
It can be achieved as mentioned in the below examples -

1.
isql -S -U -P << EOF

EOF

Use isql command with the aforesaid options.
<< EOF says, do the execution of the embedded sql after this command line till word "EOF" is encountered.

The output would be displayed on command prompt only.


2. isql -S -U -P << EOF >analytics_result.txt

EOF


Same as above with one addition , the output can bbe redirected to a file with indirection ">". Here the output file is "analytics_result.txt"


3.
isql -S -U -P -i analytics_query.sql << EOF
EOF


In case of big queries or multiple query execution, all sql stuffs could be packaged into a sql file and the file path could be mentioned with the option "-i".


Scope -

This automation could be used for database functionalities and running database queries as a scheduled job.




Avaneesh
07-01-2009