Opinion

As in all other professions, ethical conduct in software development is a must

As part of my degree coursework back in 2002, I had to write a whitepaper that outlined the commonsense principles behind ethical conduct within the sphere of IT, as affairs stood at that time. A recent conversation I had about the ethical conduct of software engineers prompted me to revisit that old paper, and try to extend the principles that I had outlined in it, to address the ethical issues that may arise when developing software applications. I’d like to highlight the underlying principles that ought to govern the good conduct of software engineers, for the benefit of newbies to this noble profession.

Whilst some folks might put forward issues such as violation of privacy, infringement of ownership rights or plain malicious intent (such as developing viruses) on top of their list of ethical concerns facing software developers, I’d like to begin with the issue of professional integrity as a developer. Basically, a software developer must be committed to building a safe, useful, reliable and secure piece of software. This means that one must:

  • Not build malware unless it is for the explicit purpose of testing the security of an application in a controlled environment
  • Have a rational use case for the application being developed; it must not be a piece of junk
  • Test the software thoroughly and be committed to fixing defects in it
  • Ensure to the best of one’s ability, that any information stored within the application should available only to those who are authorized to view it, as per the business use case
  • Learn and follow recognized engineering practices in the industry, with respect to the architecture, design, coding, testing and distribution of the software. In other words, one must be a craftsman and not a hacker
  • Be confident that the application’s business use case doesn’t violate any law of the land, civil or criminal. Building a piece of software that facilitates a Ponzi scheme would be an example of unethical conduct
  • Report to the client or the employer promptly if in one’s opinion, a project is likely to fail, to prove too expensive, to violate intellectual property law, or otherwise to be problematic

A developer who fails to adopt the above guiding principles into his code of conduct would end up being “unprofessional”  by today’s standards.

Let us now consider the issue of respecting the intellectual property rights of others. The parameters of this issue have been debated for many years, sometimes with winning arguments emerging from the flipside such as students in developing countries being unable to afford software development tools due to the absence of pricing structures catering to their poor economic conditions. However this conflict seems largely behind us now because big companies building popular propriety development tools (such as Microsoft) have created special pricing mechanisms to address the issue of global affordability and affordability for students. Also the vast open source development movement sprang about to address this very problem; and today we can say that, at least in principle, any developer can choose a toolset for building powerful, robust applications at zero infrastructure cost. Therefore the ethical principles behind respect for IP rights become even more morally binding, such as:

  • Acknowledging borrowed IP that is being used as subcomponents of your application. This is particularly true for some free or open source apps, frameworks or code snippets. The providers of such software usually state the requirements for acknowledgement of their IP rights
  • Adhering to the detailed parameters of licensing agreements of all software at all times. We should not exploit loopholes that allow us to gain access to the working tools, and then exceed parameters that the provider has stipulated but which he is unable to control, such as limits on concurrent database access. The providers have presumably decided on a fair business model for the software tools they peddle based on their operating costs, so we must not cheat them
  • Helping those who have helped us, such as submitting honest reviews about the software subcomponents we have leveraged. This again is particularly true for open source frameworks; actively participate in ironing out issues in them and campaign for their progress
  • Having clear licensing agreements for all applications we develop, and ensuring they are read by users when they install or sign up for your product

Another interesting aspect to ethical conduct in software development is the fair management of projects, to cater to the best interest of all parties concerned; namely the client, the developer(s) and the organization that employs their services for profit. This issue is mainly present in service companies that employ large teams of software developers to build applications for clients rapidly. Some of the recognized fair management practices include:

  • A pragmatic estimation of the effort involved and delivery schedule, and an honest communication of the same to the client
  • Close collaboration between the client and the developers, and ensuring transparency on both sides such as technical difficulties or changes in user expectations
  • Monitoring of the ongoing effort and correcting capacity inadequacies proactively. Whilst enterprise software development is sometimes tedious and may require overtime effort from developers, the long-term goal of a ethical project manager would be to match client expectations with available capacity, and ensure the developers enjoy a work-life balance
  • Develop a firm opinion about the risks and issues arising in a development project, based on investigation and past experience, and take action to mitigate the problems at hand. In other words, an ethical project manager would have a “backbone” to make decisions and influence people
  • Ensure that whilst team communication can be assertive or relaxed based on the situation at hand, it always remains professional and follows all norms of the communication decency

There are many other ethical issues that we may face as software engineers, please see below some interesting references that cover this topic more thoroughly.

When we set standards for a profession, we draw a line on the sand between the acceptable and the unacceptable, which sets the stage for greater respect and higher compensation. I invite you all to choose the higher standard.

References:
Computer Ethics Institute:http://computerethicsinstitute.org/
Communication Decency Act: http://en.wikipedia.org/wiki/Communications_Decency_Act
Software Engineering Code of Ethics and Professional Practice:
http://www.acm.org/about/se-code

Opinion

Django, for better performing websites with rapid development

Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design.Django lets the developers build deep, dynamic, interesting sites in an extremely short time. The framework is designed to let the developer focus on the fun, interesting parts of the job while easing the pain of the repetitive bits. In doing so, it provides high-level abstractions of common Web development patterns, shortcuts for frequent programming tasks, and clear conventions on how to solve problems. At the same time, Django tries to stay out of the way, letting the developer work outside the scope of the framework as needed.

Django is usually called an MVC framework, and justifiably so. It is very heavily influenced by classical MVC and it’s even possible to argue that Django improves the architectural pattern. In Django, the three core layers are the Model, the View, and the Template. A key advantage of such an approach is that components are loosely coupled.

Having come to know of the interesting features of Django, we recently worked on a pilot product development project, called Xaffo, which had demanding needs for a web framework. Xaffo, a cloud-based social media monitoring tool,allows users to analyze the popularity of their brands among leading social networks.

Why we decided that Xaffo needs a web framework like Django
Xaffo basically deals with large chunks of social media analysis data shuttled between different web services, with huge task lists in the background that requires higher performance and scalability.The Xaffo prototype was initially built on Google App Engine and therefore in Python,and keeping the python code was also an essential part in our decision-making.

Meeting performance expectations when handling large sets of data was the challenge where features of Django became relevant and interesting. Handling large number of tasks in the background, Celery (explained below) came in handy to support Django to dynamically add or remove workers to handle the tasks.

On account of handling large sets of data, Django makes a better pair with MongoDB, which brings out easy database connectivity with high performance.

Xaffo is hosted behind nginx web server with uWSGI for high performance and static and dynamic content serving.The combination of this hosting environment and Django proved optimal. Xaffo is hosted on Amazon EC2 achieving the scalability where Django becomes the perfect web framework to match all these components.

Django made it easy to achieve tedious tasks that Xaffo demanded, with its appealing features such as,

  • Object Relational Mapping
  • Template System
  • URL Resolver
  • Forms
  • Admin Site

In addition to the aforementioned features of Django, Xaffo relies largely on Django’s modularity.
We used several 3rd party Django packages, such as:

  1. Celery
  2. MongoEngine
  3. Flower

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.The execution units, called tasks, are executed concurrently on a single or multiple worker servers using multiprocessing, Eventlet, or gevent. Tasks can be executed asynchronously (in the background) or synchronously (wait until ready).Celery is used in production systems to process millions of tasks a day.

Celery is used with Xaffo to manage the periodic tasks that are executed to fetch and calculate data gathered from various social network APIs. RabbitMQ is used as the message broker.

MongoEngine is an object document mapper for Django, which is very similar to Django’s own ORM. This enables developers who are already familiar with the Django ORM to interact with MongoDB without having to go through a whole new API documentation.
Celery Flower is a tool used with Xaffo to monitor the periodic tasks executed by Celery. This tool provides a web interface with information such as task progress, graphs and statistics.

To wind up, Xaffo was a successful project with Django, proving to us that Django web framework is highly suitable for projects that require higher performance,scalability, and high load of backend processing with large chunks of data.Its also a RAD framework in this context, and facilitates meeting tough deadlines.

Django takes away the tedious tasks of the development environment and makes it easier to build better web apps more quickly with less code. We at Calcey recommend it wholeheartedly!

More info:
Django – http://www.djangoproject.com
MongoDB – http://mongoengine.org/
Celery –http://celeryproject.org/
Flower – http://docs.celeryproject.org/en/latest/userguide/monitoring.html#flower-real-time-celery-web-monito
RabbitMQ- http://www.rabbitmq.com/features.html

Life at CalceyOpinion

Calcey Technologies adopts a domain-driven design approach

Domain-Driven Design (DDD) is an object-oriented approach to designing software based on the business domain, its elements and behaviours, and the relationships between them. It aims to build software systems that are a realization of the underlying business domain, by defining a domain model1 expressed in the language of business domain experts.

The core idea is that the business domain stakeholders and the technical team must communicate in a ubiquitous language. A domain model can be viewed as a framework from which different solutions can then be rationalized. For example, the domain might be retail, and three different solutions sitting on the domain model for retail sales might be:

  1. An online store for the general public.
  2. An order processing system for the store’s staff.
  3. A special offers app for mobile devices, that notifies customers about offers based on proximity to the store.

The domain model will be defined with the assistance of experts in the retail business, where certain “fundamental” concepts in the retail trade will be built into the model as entities, value objects and aggregates. These entities will reside within a domain layer in the conceptual architecture of the overall system, to be leveraged by upper layers that render end-user functionality.

We recently adopted a domain-driven design approach to build an app for the centralized distribution and control of Multimedia Marketing Content to Sales Staff. The business domain is one of marketing content management, and we sought our client’s expertise in this field to help build a generic domain model.

In marketing content management, the basic concept is that there are market segments (aka business units), and marketing content is associated with these segments. The actual content can be folders or multimedia content items. In domain-driven design, your objective is to create a model of the domain. You need to identify what are the items (entities) you need to accomplish the desired functionalities of your application. You need to identify the relationships among different entities and how they interact among themselves. You need to find if the business goal of your client is achievable using your domain model. You do not need to know how and where the data of your domain will persist or even if the data do need to persist while you do the model of the domain.

This ignorance about your persistence medium will make your domain model free from any coupling with the persistence layer of the application. This will eventually separate the concerns of persistence and its communication mechanism from your domain model. As a result, your application will be free from coupling with any data store and will be very easily unit testable.

Of course, in a real application, you do need to have a database. But your domain model will have no knowledge about that. All it will know is the existence of a “repository”  that will eventually manage your application’s persistence concerns.

I hope I was able to provide a teeny insight into what DDD is about. Eric Evans popularized the DDD approach by presenting this concept in “Domain-Driven Design: Tackling Complexity in the Heart of Software”. I strongly recommend this excellent book to all budding software architects. Here is a diagram from Evans’ work, describing the key patterns involved.

Let us part with this thought; imagine, once your code becomes readable to those familiar with the business domain, both peer review and knowledge transfer would become a whole lot easier.

  1. Domain-driven architectural style (definition by Microsoft): http://msdn.microsoft.com/en-us/library/ee658117.aspx#DomainModelStyle
  2. Domain-driven design by Eric Evans: http://www.amazon.com/Domain-Driven-Design-Tackling-Complexity-Software/dp/0321125215
  3. A quick refresher on DDD: http://www.codeproject.com/Articles/339725/Domain-Driven-Design-Clear-Your-Concepts-Before-Yo