Ratnesh Dubey

Thinking Models: Navigating Complexity in Software Engineering

My journey into the world of thinking models began unexpectedly. It wasn’t until I delved into the book “The Great Mental Model” recently that I discovered the concept of thinking models. This sparked my curiosity to explore what thinking models exist specifically within the realm of software engineering. Let’s embark on this exploration together, diving into some of these models and illustrating their impact with real-world examples in the domain of software engineering.

Systems Thinking: Seeing the Forest for the Trees

Understanding the Whole System: Systems thinking compels us to look beyond individual components of a software system to see the entire ecosystem. It’s about understanding how parts interact, influence one another, and contribute to the system’s overall behavior.

Real-World Application: Consider the optimization of a cloud-based storage service. By employing systems thinking, engineers identified that the bottleneck wasn’t just the storage retrieval speed, but also the way data was replicated and cached across regions. By addressing these interrelated components as a whole, they significantly improved data retrieval times and user satisfaction.

The Pareto Principle: Focusing on the Vital Few

Maximizing Impact with Minimal Effort: The Pareto Principle, or the 80/20 rule, suggests that a small number of causes often lead to a large portion of the effects. In software engineering, this means identifying the critical features or bugs that will have the most significant impact on performance or user experience.

Real-World Application: A social media platform used the Pareto Principle to enhance its moderation system. By analyzing user reports, they discovered that 20% of the content types were responsible for 80% of the community’s complaints. Targeting these content types for stricter moderation policies resulted in a significant decrease in user complaints.

First Principles Thinking: Building from the Ground Up

Questioning Assumptions for Innovative Solutions: First principles thinking involves breaking down complex problems into their most basic elements and reassembling them in new, innovative ways. It encourages questioning assumptions and approaching problems from a foundational perspective.

Real-World Application: In developing a new encryption algorithm, a team applied first principles thinking to challenge existing assumptions about cryptographic security and computational efficiency. This approach led them to develop a novel algorithm that was not only more secure but also more efficient, revolutionizing data protection strategies in their application.

The OODA Loop: Staying Agile in Decision Making

Cycle of Observation and Action: The OODA Loop, standing for Observe, Orient, Decide, Act, is a model developed for quick, effective decision-making. It’s about constantly assessing the situation, making decisions, and adapting to outcomes—a perfect fit for the agile nature of software development.

Real-World Application: A mobile game development team applied the OODA Loop to rapidly iterate on their game’s design based on player feedback. By quickly observing player behaviors, orienting their strategy accordingly, deciding on the changes, and implementing them, they were able to significantly improve player engagement and retention.

Computational Thinking: Problem Solving Like a Computer Scientist

Systematic Approach to Problem Solving: Computational thinking is about approaching problems in a way that a computer might solve them. It involves skills such as decomposition, recognizing patterns, abstracting, and creating algorithms.

Real-World Application: Faced with the challenge of improving search functionality in a large database, a team employed computational thinking to decompose the problem, identify inefficient search patterns, and abstract these patterns into a more efficient search algorithm. The result was a dramatic reduction in search times and an improvement in user experience.

Embracing Diverse Thinking Models

The application of these thinking models in software engineering not only broadens our approach to solving problems but also ignites creativity, improves efficiency, and enhances our ability to develop solutions that truly meet user needs. By adopting models like Systems Thinking, the Pareto Principle, First Principles Thinking, the OODA Loop, and Computational Thinking, engineers can navigate the complexities of software development with greater agility and inventiveness.

As the field of software engineering continues to advance, embracing these diverse thinking models will be crucial for staying ahead of the curve. They empower us to dissect intricate challenges, foster innovation, and ultimately, craft software solutions that are not only technically robust but also deeply impactful.

Common RESTful API Design Mistakes and Their Remedies

Designing RESTful APIs is an art that requires attention to detail, a deep understanding of HTTP, and a commitment to the principles laid out by Roy Fielding. However, many APIs that claim to be RESTful fall short of these principles. In this blog, let’s explore 20 common mistakes and their remedies :

1. Misuse of HTTP Methods

Non-RESTful: POST /getSubscription?id=123
RESTful Remedy: GET /subscriptions/123

2. Ignoring Caching Opportunities

Non-RESTful: Responses without Cache-Control.
RESTful Remedy: Include Cache-Control: max-age=3600 in GET responses for subscriptions.

3. Inconsistent Error Handling

Non-RESTful: {"error": "It failed"}
RESTful Remedy: {"error": {"code": 404, "message": "Subscription not found"}} with a 404 status code.

4. Lack of Resource Nesting

Non-RESTful: /subscriptions, /users
RESTful Remedy: /users/123/subscriptions to list subscriptions for user 123.

5. Overuse of Query Parameters

Non-RESTful: /subscriptions?userId=123
RESTful Remedy: /users/123/subscriptions

6. Forgetting Pagination

Non-RESTful: Returning all subscriptions in one call.
RESTful Remedy: /subscriptions?page=1&limit=10

7. Failing to Version the API

Non-RESTful: Directly modifying the API structure.
RESTful Remedy: /v2/subscriptions/123

8. Underutilizing HTTP Headers

Non-RESTful: Passing API keys as query parameters.
RESTful Remedy: Authorization: Bearer <token>

9. Not Supporting Conditional Requests

Non-RESTful: Ignoring use of ETag or Last-Modified.
RESTful Remedy: Respond with ETag header and honor If-None-Match requests.

10. Poor Resource Identification

Non-RESTful: Ambiguous identifiers.
RESTful Remedy: /subscriptions/{uuid} where uuid is a unique identifier.

11. Ignoring Idempotency in Non-GET Requests

Non-RESTful: POST /subscriptions/123/renew leading to multiple renewals.
RESTful Remedy: PUT /subscriptions/123/renewal to renew idempotently.

12. Not Leveraging HATEOAS

Non-RESTful: { "subscriptionId": "123", "status": "active" }
RESTful Remedy:

{ "subscriptionId": "123", "status": "active", "_links": { "self": { "href": "/subscriptions/123" }, "cancel": { "href": "/subscriptions/123/cancel" } } }

13. Mixing Singular and Plural Nouns

Non-RESTful: Inconsistent use of singular and plural in resource paths.
RESTful Remedy: Consistently use plural nouns for resource names: /subscriptions for collection routes and /subscriptions/{id} for specific entities.

14. Exposing Internal Implementation Details

Non-RESTful: Revealing database IDs or internal structure through APIs.
RESTful Remedy: Use abstracted, meaningful identifiers: /subscriptions/12345 where 12345 is an opaque identifier.

15. Not Providing Filtering, Sorting, or Searching Capabilities

Non-RESTful: Forcing clients to download entire data sets.
RESTful Remedy: Implement query parameters for server-side operations: /subscriptions?status=active&sortBy=startDate

16. Lack of Media Type Negotiation

Non-RESTful: Only supporting application/json.
RESTful Remedy: Use the Accept header for format requests. Respond to Accept: application/xml for XML format.

17. Incomplete or Missing Documentation

Non-RESTful: APIs without comprehensive documentation.
RESTful Remedy: Provide detailed, up-to-date documentation portal.

18. Not Utilizing PATCH for Partial Updates

Non-RESTful: Using PUT for all updates.
RESTful Remedy: Implement PATCH for partial resource updates.

19. Treating Collections and Individual Resources the Same

Non-RESTful: Same endpoint for collection and individual resource operations.
RESTful Remedy: Differentiate endpoints for collections and individual resources: POST /subscriptions for creating and GET /subscriptions/123 for fetching.

20. Not Handling Asynchronous Operations Properly

Non-RESTful: Expecting synchronous completion of long-running tasks.
RESTful Remedy: Use asynchronous patterns with initial 202 Accepted responses: POST /subscriptions/123/renew returns 202 Accepted with a Location header for status checks.

The Strangler Pattern: A Pragmatic Approach to Modernizing Legacy Applications

Introduction

Updating a legacy software system is a critical challenge many businesses face. It’s about enhancing what works while introducing new, efficient technologies. This is where the Strangler Pattern shines. It offers a structured yet flexible way to modernize systems without the need for a complete overhaul from the start. Let’s explore how this approach makes the complex process of software upgrade more manageable and less risky.

The Legacy Software Dilemma

Legacy systems are often the bedrock of a business but come with their set of problems. They might be built on outdated technologies that are hard to integrate with newer systems, and their monolithic nature can make any change a risky venture. The challenge lies in updating these systems without disrupting the daily flow of business operations.

Key Challenges in Legacy Systems

Outdated Technologies: Systems built on older frameworks struggle to integrate with newer, more efficient technologies.
Monolithic Architecture: Changes in one part of the system can have unforeseen effects on others, complicating updates and bug fixes.
Integration Complexities: Bridging the gap between older systems and modern development practices is often not straightforward.

Implementing the Strangler Pattern

Module Identification: The first step is to identify independent modules within the application that can be updated separately.
Parallel Development: Develop new features alongside the existing system. This is akin to building a new structure next to an old one.
Redirecting Traffic: Gradually shift the user interactions from the old components to the new ones, ensuring the new system is reliable before fully transitioning.
Phasing Out Old Components: Once the new system is stable, start retiring the old components.
Repeat the Process: Continue this process, one module at a time, until the entire system is updated.

A Real-World Example

Consider modernizing the user authentication system of an e-commerce platform. Instead of revamping the entire platform, the authentication module is rebuilt using modern standards and gradually integrated into the existing system. This way, the platform remains operational while the module is updated.

Conclusion

The Strangler Pattern offers a strategic way to modernize legacy systems. By focusing on one component at a time and ensuring each new piece works seamlessly before moving on to the next, it reduces risk and allows for a smoother transition to modern technologies. It’s a practical approach for businesses looking to update their software systems efficiently and effectively.

How to replace if-else with polymorphism

In object-oriented programming, the use of if-else statements can quickly make code difficult to read, understand and maintain. As the number of conditions and their complexity increases, so does the potential for errors and bugs. Polymorphism, on the other hand, offers a more elegant and efficient solution to this problem. By allowing objects of different classes to be treated as objects of a common superclass, polymorphism can simplify the code, making it more readable and extendable. In this article, we will explore how to improve if-else statements with polymorphism and how it can help you write better and more maintainable code. Let us understand this with examples:-

class ShapeCalculator {
    public double CalculateArea(Shape shape) {
        if (shape is Circle) {
            Circle c = (Circle) shape;
            return Math.PI * c.radius * c.radius;
        } else if (shape is Rectangle) {
            Rectangle r = (Rectangle) shape;
            return r.width * r.height;
        }
        return 0.0;
    }
}

In this example, the CalculateArea method uses an if-else statement to check the type of the shape object and then calls the appropriate calculation method for that type. This approach works, but it’s less maintainable and extendable than using polymorphism.

For example if a new shape ‘Square’ is added to the code, we will have to add an additional ‘else if’ block to handle that. And every time a new shape is added, we will have to modify the CalculateArea method which can lead to errors and increased complexity in the code. On the other hand, with polymorphism, as every class knows how to calculate its area, this problem can be avoided.

class Shape {
    public virtual double Area() {
        return 0.0;
    }
}

class Circle : Shape {
    private double radius;
    public Circle(double r) {
        radius = r;
    }
    public override double Area() {
        return Math.PI * radius * radius;
    }
}

class Rectangle : Shape {
    private double width;
    private double height;
    public Rectangle(double w, double h) {
        width = w;
        height = h;
    }
    public override double Area() {
        return width * height;
    }
}

class Triangle : Shape {
    private double base;
    private double height;
    public Triangle(double b, double h) {
        base = b;
        height = h;
    }
    public override double Area() {
        return 0.5 * base * height;
    }
}

Now we can use this class to calculate the area of any shape, whether it’s a circle, rectangle, or triangle, without having to use any if-else statement. Because each shape knows how to calculate its own area through the Area method, the CalculateArea method can simply call the Area method on the passed-in shape object, regardless of its actual type.

The use of polymorphism here allows the calculation of area to be done in a more maintainable and extendable way. As it allows you to add new shape with new area calculation method without modifying the existing code.

In conclusion, polymorphism allows us to replace a series of if-else statements with virtual functions, making our code more readable, maintainable and extendable.

Five React must-follow practices

Here are five best practices for working with React:

1.Use functional components whenever possible: Functional components are simpler and easier to understand than class-based components, and they can often do everything that class-based components can do. For example:

2. Use the useEffect hook for side effects: The useEffect hook allows you to perform side effects (such as API calls or subscribing to events) in functional components. It is a safer and more efficient alternative to the componentDidMount, componentDidUpdate, and componentWillUnmount lifecycle methods. For example:

3. Use the useMemo hook for performance optimization: The useMemo hook allows you to memoize the result of expensive calculations so that they are only recalculated when the input dependencies change. This can improve the performance of your application by avoiding unnecessary re-renders. For example:

4. Use the useCallback hook to avoid unnecessary re-renders: The useCallback hook allows you to return a memoized callback function that will not change unless one of its dependencies changes. This can be useful for avoiding unnecessary re-renders when passing callback props to child components. For example:

5. Use the React.Fragment component to avoid unnecessary DOM nodes: The React.Fragment component allows you to return multiple elements from a component without adding an extra DOM node. This can be useful for keeping your DOM structure clean and avoid issues with CSS or other DOM manipulation. For example:

Patterns for designing self healing apps

Self healing is important concern while developing an application. For example, if a downstream service is not available , how can the app handle this situation? Will it retry more for the service which is already down or will it understand the situation and stop hammering a failing service. What if there is failure in one subsystem which can sometime cascade, for example a thread or a socket not getting freed in timely manner can result to cascading failures. All too often, success path is well tested but not the failure path. There are many patterns for handling these failures but here are few must have pattern to gracefully handle these situation.

Pattern	Premise	Aka	How does the pattern mitigate?
Retry failed operations with retry strategy (Retry Pattern) Implementation Detail	Many faults are transient and may self-correct after a short delay. Have a retry pattern with strategy of increasing delay.	“Maybe it’s just a blip”	Allows configuring automatic retries.
Protecting failing app with Circuit Breaker (Circuit-breaker) Implementation Detail	When a system is seriously struggling, failing fast is better than making users/callers wait. Protecting a faulting system from overload can help it recover.	“Stop doing it if it hurts” “Give that system a break”	Breaks the circuit (blocks executions) for a period, when faults exceed some pre-configured threshold.
Better caller experience with timeout (Timeout) Implementation Detail	Beyond a certain wait, a success result is unlikely.	“Don’t wait forever”	Guarantees the caller won’t have to wait beyond the timeout.
Isolate critical resources (Bulkhead Isolation) Implementation Detail	A Ship should not sink because there is hole in one place. When a process faults, multiple failing calls can stack up (if unbounded) and can easily swamp resource (threads/ CPU/ memory) in a host. This can affect performance more widely by starving other operations of resource, bringing down the host, or causing cascading failures upstream.	“One fault shouldn’t sink the whole ship”	Constrains the governed actions to a fixed-size resource pool, isolating their potential to affect others.
Throttle clients with Rate Limit (Rate-limit) Implementation Detail	Limiting the rate a system handles requests is another way to control load. This can apply to the way your system accepts incoming calls, and/or to the way you call downstream services.	“Slow down a bit, will you?”	Constrains executions to not exceed a certain rate.
Fallback Implementation Detail	Things will still fail – plan what you will do when that happens.	“Degrade gracefully”	Defines an alternative value to be returned (or action to be executed) on failure.

What’s wrong with just hashing a password?

Storing password is critical for any application. If you do not take right precaution then you loose your user password to attacker. For password security, storing password in plain text in database is certainly a bad design. Hashing password is well known but unfortunately it is also not enough. As we know that when user tries to log in, the hash of the password they entered is checked against the hash of their password in the database. If the hash matches, the user gains access to the account. If an attacker gains access to password database, they can use the rainbow table attack to compare hashed passwords to potential hashes in the table. The rainbow table then gives plain text possibilities with each hash, which the attacker can use to access an account. For example, if attacker has a rainbow table with the hash for the password “welcome123” any user that uses that password will have the same hash, so that password can easily be cracked.

To mitigate this attack, we use password salting. As per OWASP “a salt is a unique, randomly generated string that is added to each password as part of the hashing process”.

The password in the database can be stored in the following format Hash(password + salt). A salt randomizes each hash by adding random data that is unique to each user to their password hash, so even the same password has a unique hash. If someone tried to compare hashes in a rainbow table to those in a database, none of the hashes would match, even if the passwords were the same.

Nonetheless, rainbow tables may not be the biggest threat to organizations today. Still, they are certainly a threat and should be considered and accounted for as part of an overall security strategy.

5 Tech Talks Every Software Engineer should watch

I love watching tech talks. I’ve compiled a list of must-see tech talks for every software engineer. However, this list doesn’t have any category on the basis of programming language or platform instead it focuses on high level general topics: