Learning
  • Software Engineering Golden Treasury
  • Trail Map
  • Caching
    • Alternatives to use before using cache
    • Caching Architecture
    • Cache Invalidation and Eviction
    • Cache Patterns
    • Cache
    • Consistency
    • Distributed Caching
    • Issues with caching
    • Types of caches
  • Career
    • algo types
    • Backend Knowledge
    • Burnout
    • consultancy
    • dev-level
    • Enterprise Developer
    • how-to-get-in-tech-from-other-job
    • how-to-get-into-junior-dev-position
    • induction
    • Interview
    • junior
    • mid
    • New Job
    • paths
    • Principle/staff Engineer
    • Requirements for job
    • Senior Dev capabilities
    • learning
      • automating-beginner
      • company1
        • analyst-progression
        • core-eng-progression
        • dev-progression
        • perf-eng-progression
        • soft-deliv-progression
    • mentoring
      • mentor-resources
    • recruitment
      • questions
      • Spotting posers
  • Computer Science
    • boolean-algebra
    • Compiler
    • Finite State Machine
    • Hashing
    • Algorithms
      • Breadth Firth Search
      • complexity
      • Depth First Search
      • efficiency
      • Sliding Window
      • sorting
    • data-structures
      • AVL Trees
      • data-structures
      • Linked List
    • machines
      • Intel Machine
      • Turing Machine
      • von neumann machine
      • Zeus Machine
  • devops
    • The 5 Ideals
    • microservice
    • Artifact repository
    • Bugs and Fixes
    • Build police
    • cloud-servers
    • Deployments
    • Environments
    • GitOps
    • handling-releases
    • infrastructure-as-code
    • System Migrations
    • SDP
    • On Premises Hosting
    • Properties/configuration
    • Release process
    • Release
    • Roll Outs
    • serverless
    • Serverless
    • Cloud Services
    • Versioning
    • AWS
      • deploy-docker-esc
      • cloud-practitiioner-essentials-notes
        • Module 1 - Intro to AWS
        • Module 2 Compute in the cloud
        • Module 3 Global Infrastructure and Reliability
        • Module 4 Networking
        • Module 5 Storage and Databases
        • Security
        • 7 Monitoring and Aanlytics
        • 8 Pricing and Support
        • 9 Migration and Innovation
      • developer-associate
        • AWS Elastic Beanstalk
    • build-tools
      • Managing dependecies
      • Apache ANT
      • Gradle
        • Custom Plugins
        • local-jars
      • Project Management - maven
        • Archtypes
        • Build Lifecycles
        • Customising build lifecycle
        • Dependencies
        • Directory layout
        • jar-files
        • one-to-one
        • Modules
        • Phases
        • Maven Plugins
        • POM
        • profiles
        • setup
        • Starting a maven project
        • wrapper
    • CI/CD
      • Continuous Delivery
      • zookeeper
      • Continuous Integration (CI)
      • github-actions
      • Pipeline
      • Teamcity
    • Cloud computing
      • Overview
      • Service Models
      • Cloud Services
    • containers
      • Best Practices
      • Docker
    • Infrastructure
      • IT Infrastructure Model
      • Non functional Attributes (Quality Attributes)
        • Infrastructure Availability
        • Performance
        • Secruity
    • monitoring
      • Alerting
      • Monitoring & Metrics
      • Metrics
      • Ready pages
      • Splunk
      • Status pages
      • notes-devops-talk
      • logging
        • logging
        • issues
        • Logging
        • Logging
    • Service mesh
      • Service Discovery
      • Istio
    • Terraform
    • container-management
      • Kubernetes
        • commands-glossary
        • OLTP
        • config-maps
        • Links
        • ingress
        • SDP
        • minikube
        • filter
        • indexes
        • sidecar
        • continuous-deployment
  • General Paradigms
    • CAP theorem
    • designing data-intensive applications summary
    • a-philosophy-of-software-design-notes
    • Aspect oriented Programming (AOP)
    • Best Practice
    • Cargo Cult
    • Clean Code
    • Coding reflections
    • Cognitive Complexity
    • Complexity
    • Conventions
    • Design discussions
    • Design
    • Error Handling Checklist
    • Exceptions
    • Feature Flags/toggle
    • Functional requirements
    • Last Responsible Moment
    • Lock In
    • Named Arguments
    • Naming
    • Performance Fallacy
    • Quality
    • Redesign of a system
    • Resuse vs Decoupling
    • Rules for software designs
    • Sad Paths
    • Scaling Webservices
    • Scientific Method
    • stream-processing
    • Upstream and Downstream
    • Patterns
      • Client-SDK-Pattern
      • ORM
      • Api gateway
      • Business Rules Engine
      • cache
      • Composition Root
      • Dependency Injection Containers
      • Dependency Injections
      • Double Dispatch
      • Exception Handling
      • Gateway pattern
      • Humble Object
      • Inheritance for reuse
      • Null Object Pattern
      • Object Mother
      • Patterns
      • Collection pipeline pattern
      • Service Locator
      • Setter constructor
      • Static factory method
      • Step Builder Pattern
      • telescopic constructors
      • Toggles
      • API
        • Aims of API designs
        • Avoid Checked Exceptions
        • Avoid returning nulls
        • Be defensive with your data
        • convience-methods
        • Fluent Interfaces
        • Loan Pattern
        • prefer-enums-to-boolean-returns
        • return-meaningful-types
        • Small intefaces
        • Support Lambdas
        • Weakest type
      • Gang of Four
        • Builder
        • Factory Pattern
        • Strategy Pattern
        • Template
        • abstract Factory
        • Adapter
        • Bridge Pattern
        • Chain of responsibility
        • Command Pattern
        • Composite Design Pattern
        • Decorator Pattern
        • Facade Pattern
        • Flyweight pattern
        • Guard Clause
        • Interpreter
        • html
        • Mediator Pattern
        • Memento Pattern
        • Observer
        • Prototype
        • Proxy
        • Singleton
        • State Pattern
        • Visitor Pattern
    • Architecture
      • Entity Component System
      • Integration Operation Segregation Principle
      • Adaptable Architecture
      • Architecture
      • C4 Modelling
      • cell-based
      • Clean/Hexagonal Architecture
      • Codifying architecture
      • Correct By configuration
      • Cost Base Architecture
      • Data Oriented Design
      • deliberate
      • Domain oriented DOMA
      • Event Driven Architecture
      • Evolutionary Architecture
      • examples
      • Feature Architecture
      • Framework and Libraries
      • functional-core-imperative-shell
      • Layered Architecture
      • Micro services
      • monoliths-to-services
      • Multi tiered Architecture
      • Multi tenant application
      • Resilient Architecture
      • stage event driven architecture (SEDA)
      • links spring rest app
      • Tomato Architecture
      • Tooling
      • Types of architecture
      • checklist
        • Checklist for new project
        • Back end Architecture Checklist
        • Front end Architecture Checklist
        • Mobile Architecture Checklist
      • Cloud Patterns
        • Command and Query Responsibility Segregation (CQRS)
        • Event Sourcing & CQRS
        • Asynchronous Request and Reply
        • Circuit Breaker
        • Retry
        • Sidecar
        • Strangler pattern
      • Domain driven design
        • value & entity
      • Microservices
        • Alternatives to choosing microservices first when scaling
        • Consistency in distributed systems
        • 12 Factor applications
      • Modularity
        • Module monolith vs Microservices
        • Spring Moduilth
      • Architecture Patterns
        • Hexagonal architecture
        • Inverting dependencies
        • Layering & Dependency Inversion Principle
        • Mappings
        • Vertical Slice architecture
        • Web Client Server
        • domain
          • Business and Data Layers Separation
          • DTO
          • Domain Model Pattern
          • Domain Object
          • Transaction Script/ Use Case pattern
        • Enterprise Patterns
          • Concurrency
          • Distribution strategies
          • Domain layer patterns
          • Layering/organisation of code
          • Mapping to datasource
          • Session State
        • Usecases
          • Use case return types
      • Serverless
        • Knative
    • Design architecture aims
      • back of envelope
      • Design ideas
      • Design mistakes
      • high-volume-design
      • ISO Quality Attributes
      • Non functional requirements
      • “Designing for Performance” by Martin Thompson
      • High Performance
      • Qaulity Attributes
        • Availability
        • System Availability
        • Fault Tolerance
        • interoperability
        • Latency
        • Maintability
        • Modifiability
        • Performance
        • Readability
        • Reliability
        • Scalability vs performance
        • Scalability
        • Scaling
        • statelessness
        • Testability
        • Throughput
      • System Design
      • web-scalability-distributed-arch
        • scalable-and-distributed-web-architecture
    • README
      • Conflict-free Replicated Data Type
      • Fallacies
      • Load balancing
      • Rate Limiting
      • Transactions
    • Patterns of Enterprise Application Architecture
      • Repository Pattern
      • Rules Engines
      • scatter-gather
      • Specification Design Pattern
      • Table Driven Development
      • Workflow Design Patterns
        • Triggers
    • Principles
      • Do It Or Get Bitten In The End
      • Dont Repeat Yourself
      • Habitability
      • Keep it simple
      • Responsibility Driven Design
      • Ya Ain’t Gonna Need It
      • Conceptual Overhead
      • CUPID
      • Reuse existing interfaces
      • Facts and Fallacies
      • locality of behaviour
      • Separation of Concerns
      • Simplicity
      • SLAP principle
      • Step down rule
      • Unix Philosophy
      • Wrong abstractions
      • SOLID
        • 1. Single Responsibility Principle
        • 2. Open Close Principle
        • 3. Liskov Substitution Principle
        • 4. Interface Segregation Principle
        • 5. Dependency Inversion Principle
        • GRASP (General Responsibility Assignment Software Principles)
        • Solid for packages
          • jobs
          • CCP
          • CRP
          • REP
          • egress
          • gossip-protocol
        • STUPID
    • programming-types
      • Coding to Contract/Interface
      • Links
      • Declarative vs Imperative Programming Languages
      • defensive-programming
      • Design by contract
      • Domain Specific Languages (DSL)
      • Event Driven
      • file-transfers
      • Logical Programming
      • Mutability
      • Self Healing
      • Simplicity
      • Type Driven Design
      • Value objects
      • Aspect Oriented Programming
      • Concurrent and Parallel Programming
        • Actor Model
        • Asynchronous and Synchronous Programming
        • Batch processing
        • Concurrency Models
        • SAP
        • Multithreading
        • Non Blocking IO
        • Optimistic vs Pessimistic Concurrency
        • Thread per connection or request model
        • Actor
        • aysnchronous-tasks
          • Computational Graphs
          • Divide and conquer
          • Future
          • Thread Pool
        • barriers
          • Barriers
          • Race conditions
        • design
          • agglomeration
          • Communication
          • Mapping
          • Partitioning
        • Liveness
          • Abandoned Lock
          • Deadlocks
          • Livelock
          • Starvation
        • locks
          • Read write lock
          • Reentrant lock
          • Try Lock
        • Mutual Exclusion
          • Data Races
          • Mutual Exclusion AKA Locks
        • performance
          • Amdahl's Law
          • Latency, throughput & speed
          • Measure Speed up
        • synchronization
          • Condition variable
          • producer consumer pattern
          • Semaphore
        • Threads and processes
          • Concurrent and parallel programming
          • Daemon Thread
          • Execution Scheduling
          • sequential-parallel
          • Thread Lifecycle
          • threads-and-processes
      • Functional Programming
        • Currying
        • design-patterns-to-func
        • imperative-programming
        • First class functions
        • Functional Looping
        • Higher Order Functions
        • Immutability
        • Issues with functional Programming
        • Lambda calculus
        • Lazy & Eager
        • map
        • Monad
        • Railway Programming
        • Recursion
        • Reduce
        • referential-transparacy
        • Referential transparency
        • Supplier
      • oop-design
        • Issues with object oriented code
        • Aggregation
        • Anti Patterns
        • Association
        • class-and-objects
        • Composition
        • general-laws-of-programming
        • general-notes
        • Getters and Setters
        • Inside out programming
        • Inversion of control
        • oop-design
        • Other principles
        • Outside in programming
        • Readability
        • Why OO is bad
        • README
          • abstraction
          • encapsulation
          • inheritance
          • Polymorphism
        • clean-code
          • Code Smells
          • Comments
          • Naming
          • CLEAN design
            • code is assertive
            • Cohesion
            • Connascence
            • Coupling
            • Encapsulation
            • Loose Coupling
            • Nonredundant code
      • Reactive Programming
        • reactive-programming
    • Projects and Software types
      • Applicatoin Development
      • Buying or creating software
      • Console Applications
      • Embedded Software development
      • Enterprise
      • Framework Development
      • Games
      • Library development
      • Rewriting
      • White Label Apps
    • State Machines
      • Spring State Machine
  • Other
    • 10x devs
    • Aim of software
    • Choosing Technologies
    • Coding faster
    • Component ownership
    • developer-pain-points
    • Developer Types
    • Effective Software design
    • Full Stack Developer
    • Good coder
    • Issues with Software Engineering and Engineers
    • Learning
    • Logic
    • Role
    • Software Actions
    • Software craftmanship
    • Software Designed
    • Software Engineering
    • Software
    • article-summaries
      • General notes
      • Summary of The Grug Brained Developer A layman's guide to thinking like the self-aware smol brained
      • improve-backend-engineer
      • Optimising Api
      • Simple and Easy
    • README
  • Hardware
    • Cpu memory
    • Storage
  • Integration
    • GRPC
    • API
    • Apis and communications between apps
    • asynchronous and synchronous communications
    • Batch Processing
    • Communications between apps
    • Delivery
    • Distributed Computing
    • Entry point
    • Event Source
    • SDP
    • egress
    • Graphql
    • Idempotency
    • Libraries
    • Long Polling
    • Multiplexing & Demultiplexing
    • Publish Subscribe
    • Push
    • Request & Response
    • REST
    • Remote Method Invocation
    • Remote Procedure Calls
    • Server Sent Events
    • Short Polling
    • Sidecars
    • SOAP
    • Stateless and Stateful
    • Streams
    • Third Party Integrations
    • wdsl
    • Web Services
    • Webhooks
    • repository
    • Kafka
      • Kafka Streams
    • message-queues
      • ActiveMQ
      • Dead Letter Queue
      • JMS
      • Messaging
  • Languages
    • C
    • Choosing A Language
    • cobol
    • Composite Data Types
    • creating
    • Date time
    • Numbers
    • Pass by value vs Pass by reference
    • Primitive Data Types
    • REST anti-patterns
    • Rust
    • Scripting
    • Static typing
    • string
    • Task Oriented Language
    • assembly
    • Getting started
      • Functional Concepts
    • cpp
    • Java
      • Code style
      • Garbage Collection
      • Intellij Debugging
      • Artifacts, Jars
      • Java internals
      • Java resources
      • Java versions
      • JShell
      • Libraries
      • opinionated-guide
      • Starting java
      • Java Tools
      • Why use java
      • Advanced Java
        • Annotations
        • API
        • Database and java
        • Debugging Performance
        • Files IO
        • Finalize
        • JDBC
        • jni
        • Libraries
        • Logging
        • SAP
        • Memory Management
        • Modules
        • OTher
        • Packaging Application
        • Pattern matching
        • performance
        • Properties
        • Reference
        • reflection
        • Scaling
        • Scheduling
        • secruity
        • Serilization
        • Time in Java
        • validation
        • Vector
        • Concurrency and Multithreaading
          • Akka
          • ExecutorCompletionService
          • Asynchronous Programming
          • Concurrency and Threads
          • CountDownLatch
          • Conccurrent Data Structures
          • Executor Service
          • Futures
          • reactive
          • Semaphore
          • structured concurrency
          • Threadlocal
          • Threads
          • Virtual Threads
          • Mutual Exclusion
            • Atomic
            • Synchronized
            • Thread safe class
            • Threads
        • debug
          • heap-dumps
          • thread-dumps
        • functional
          • Collectors
          • Exception Handling
          • Flatmap
          • Functional Programming
          • Generators
          • Immutability
          • issues
          • Optional
          • Parallel Streams
          • Reduce
        • networks
          • HTTP client
          • servlet-webcontainers
          • sockets
          • ssl-tls-https
      • Basics of java
        • compilation
        • computation
        • Conditonal/Flow control
        • Excuting code
        • Instructions
        • Looping/Iterating
        • memory-types-variables
        • methods
        • Printing to screen/debugging
        • Setup the system
        • Data structures
          • Arrays
          • Arayslist/list
          • Map
      • Effective Java notes
        • Creating and Destroying Objects
        • Methods Common to All Objects
        • best-practice-api
        • Classes and Interfaces
        • Enums and Annotations
        • Generics
      • framework
        • aop
        • bad
        • Dagger
        • Databases
        • Lombok
        • Mapstruct
        • netty
        • resliance4j
        • RxJava
        • Vert.x
        • Spring
          • Spring Data Repositories
          • actuator
          • cloud-native
          • H2 Db in Spring
          • Initializrs
          • JDBC Template
          • Java Persistence API (JPA)
          • kotlin
          • Pitfalls and advice
          • PRoxies
          • Reactive
          • spring security
          • spring-aop
          • Spring Boot
          • spring-jdbc
          • Spring MVC
          • Spring Testing
          • Testing
          • Transaction
          • patterns
            • Component Scan Patterns
            • Concurrency
            • Decorator Pattern in Spring
        • Micronaut
          • DI
        • Quarkus
          • database
          • Links
      • Intermediate level java
        • String Class
        • Assertions
        • Casting
        • Clonable
        • Command line arguments
        • Common Libraries/classes
        • Comparators
        • Where to store them?
        • Shallow and Deep Copy
        • Date and Time
        • Enums
        • Equals and Hashcode
        • Equals and hashcode
        • Exceptions
        • Final
        • Finally
        • Generics
        • incrementors
        • Null
        • packages and imports
        • Random numbers
        • Regex
        • Static
        • toString()
        • OOP
          • Accessors
          • Classes
          • Object Oriented Programming
          • Constructors
          • Fields/state
          • Inheritence
          • Interfaces
          • Methods/behaviour
          • Nested Classes
          • Objects
          • Static VS Instance
          • Whether to use a dependency or static method?
        • Other Collections
          • Other Collections
          • Arraylist vs Linkedlist
          • LinkedHashMap
          • Linked List
          • Priority queue
          • Sequenced Collections
          • Set
          • Shallow vs Deep Copy
          • Time Complexity of Collections
          • What Collection To use?
    • kotlin
      • Domain Specific Language
      • learning
      • Libraries
      • Personal Roadmap
      • Links
    • Nodejs
      • Performance
  • Management & Workflow
    • Agile
    • Take Breaks
    • # Communication
    • Engineering Daybook
    • Estimates
    • Feedback Loops
    • Little's law
    • Managing Others
    • poser.
    • Presentations
    • self-improvement
    • software-teams
    • Task List
    • trade-off
    • Types of devs
    • Type of work
    • Waterfall Methodology
    • coding-process
      • Bugs
      • Code Review
      • Code Reviews
      • Documentation
      • Done
      • Handover
      • Mob Programming
      • Navigate codebase
      • Pair Programming
      • Pull Requests
      • How to do a story
      • Story to code
      • Trunk based development
      • Xtreme Programming (XP)
      • debugging
        • 9 Rules of Thumb of Dubugging
        • Debugging
        • using-debugger
      • Legacy code
        • Legacy crisis
        • Working with legacy code
    • Managing work
      • Theory of constraints
      • Distributed Teams
      • estimations
      • Improving team's output
      • Kanban
      • Kick offs
      • Retrospectives
      • Scrum
      • Sign offs
      • Stand ups
      • Time bombs
      • Project management triangle
    • Notion
    • recruitment
      • In Person Test
      • Interviews
      • Unattended test
  • Networks
    • Content Delivery Network - CDN
    • DNS
    • cache control
    • Cookies and Sessions
    • Docker Networking
    • Duplex
    • Etags
    • HTTP Cache
    • HTTP - Hyper Text Transfer Protocol
    • HTTP/2
    • Http 3
    • Internet & Web
    • iptables
    • Keep alive
    • Leader Election
    • Load balancer
    • long-polling
    • Network Access Control
    • Network Address Translation (NAT)
    • Network Layers
    • Nginx
    • OSI network model
    • Persistent Connection
    • Polling
    • Proxy
    • Quic
    • reverse-proxy
    • servers
    • Server sent events (SSE)
    • SSH
    • Streaming
    • Timeouts
    • Url Encoding
    • Web sockets
    • WebRTC (Web Real-Time Communication)
    • Wireshark
    • tcp/ip
      • Congestion
      • IP - Internet Protocol
      • TCP - Transmission Control Protocol
  • Operating Systems
    • Cloud Computing
    • Distributed File Systems
    • Distributed Shared Memory
    • Input/Output Management
    • Inter-Process Communication
    • Threads and Concurrency
    • Virtualization
    • Searching using CLI
    • Bash and scripting
    • Booting of linux
    • makefile
    • Memory Management
    • Processes and Process Management
    • Scheduling
    • Scripting
    • Links
    • Ubuntu
    • Unix File System
    • User groups
    • Linux
  • Other Topics
    • Finite state machine
    • Floating point
    • Googling
    • Setup
    • Unicode
    • Machine Learning
      • Artificial Intelligence
      • Jupyter Notebook
    • Blockchain
    • Front End
      • Single Page App
      • cqrs
      • css
      • Debounce
      • Dom, Virtual Dom
      • ADP
      • htmx
      • Island Architecture
      • Why use?
      • Java and front end tech
      • mermaidjs
      • Next JS
      • javascript
        • Debounce
        • design
        • Event loop
        • testing
        • Typescript
        • react
          • Design
          • learning
          • performance
          • React JS
          • testing
      • performance
      • Static website
    • jobs
      • Tooling
      • bash text editor - vim
      • VS code
      • scaling
        • AI Assistant
        • Debugging
        • General features and tips and tricks
        • IDE - Intellij
        • Plugins
        • Spring usage
  • persistance
    • ACID - Atomicity, Consistency, Isolation, Durability
    • BASE - Basic Availability, Soft state, Eventual Consistency
    • Buffer
    • Connection pooling
    • service
    • Database Migrations - flywaydb
    • Databases
    • Eventual Consistency
    • GraphQL
    • IDs
    • indexing
    • MongoDB
    • Normalisation
    • ORacle sql
    • Partitioning
    • patterns
    • PL SQL
    • Replication and Sharding
    • Repository pattern
    • Sharding
    • Snapshot
    • Strong Consistency
    • links
    • Files
      • Areas to think of
    • hibernate
      • ORM-hibernate
    • Indexes
      • Elastisearch
    • relationships
      • many-to-many
      • SDP
      • serverless
      • x-to-x-relationships
    • sql
      • Group by
      • indexes
      • Joins
      • Common mistakes
      • operators
      • performance
    • types
      • maven-commands-on-intellij
      • in-memory-database-h2
      • Key value database/store
      • Mongo DB
      • NoSQL Databases
      • Relational Database
      • Relational Vs Document Databases
  • Security
    • OAuth
    • API Keys
    • Certificates and JKS
    • Cluster Secruity
    • Communication Between Two Applications via TLS
    • Cookies & Sessions
    • CORS - Cross-Origin Resource Sharing
    • csrf
    • Encryption and Decryption
    • Endpoint Protection
    • JWT
    • language-specific
    • OpenID
    • OWASP
    • Secrets
    • Secruity
    • Servlet authentication and Authorization
    • vault
  • Testing, Maintainablity & Debugging
    • Service-virtualization and api mocking
    • a-test-bk
    • Build Monitor
    • Builds
    • Code coverage
    • consumer-driven contract testing
    • Fixity
    • Living Documentation
    • Mocks, Stubs & Doubles
    • patterns
    • Quality Engineering
    • Reading and working with legacy code
    • Reading
    • remote-debug-intellij
    • simulator
    • Technical Debt
    • Technical Waste
    • Test cases
    • Test Data Builders
    • Test Pyramids
    • Test Types
    • Testing Good Practice
    • Testing
    • What to prime
    • What to test
    • Debugging
      • Debugging in kubernetes or Docker
    • fixing
      • How to Deal with I/O Expense
      • How to Manage Memory
      • How to Optimize Loops
      • How to Fix Performance Problems
    • Legacy Code
      • Learning
      • Legacy code
      • techniques
    • libraries
      • assertj
      • Data Faker
      • Junit
      • mockito
      • Test Containers
      • Wiremock
      • Yatspec
    • Refactoring
      • Code Smells
      • refactoring-types
      • Refactoring
      • Technical Debt
      • pyramid-of-refactoring
        • Pyramid of Refactoring
    • Test first strategies
      • Acceptance Testing Driven Developement (ATDD)
      • Behaviour Driven Development/Design - BDD
      • Inside out
      • Outside in
      • Test driven development (TDD)
    • testing
      • Acceptance tests
      • How Much Testing is Enough?
      • Approval Testing
      • Bad Testing
      • End to end tests
      • Honeycomb
      • Testing Microservices
      • Mutation testing
      • Property based testing
      • Smoke Testing
      • social-unit-tests
      • solitary-unit-tests
      • Static Analysis Test
      • Unit testing
  • Version Control - Git
    • Branch by Abstraction
    • feature-branching
    • Git patches
    • Trunk Based Development
Powered by GitBook
On this page

Was this helpful?

  1. General Paradigms
  2. programming-types
  3. Concurrent and Parallel Programming
  4. design

Communication

  • After decomposing the problem into separate tasks, the next step in our design process is to establish communication, which involves figuring out how to coordinate execution and share data between the task.

  • Some problems can be decomposed in ways that do not require tasks to share data between them.

    • Consider the job of frosting these cupcakes. If I'm tasked to add frosting to this one, and you're tasked to add frosting to that one, even though we're operating on adjacent elements in this array, there's no need for us to communicate with each other. They're completely independent tasks. This is embarrassingly easy to make parallel.

  • Sure, we could spend our quality family time together in the kitchen not talking to each other, but what if there is a need to share data between tasks? Let's say we want to decorate the cupcakes to have a rainbow pattern across them. That would require each task to know information. I need to know what color you're making your cupcakes, so I can color my cupcakes accordingly. Although our separate tasks can execute concurrently, we're not longer completely independent from each other.

    • In this type of situation, we might establish a network of direct point-to-point communication links between neighboring tasks. For each link, one task is acting as the sender, or producer of data, and the other task that needs it is the receiver or consumer.

    • That type of local point-to-point communication can work when each task only needs to communicate with a small number of other tasks.

  • But if your tasks need to communicate with a larger audience, then you should consider other structures for sharing data between multiple tasks.

    • You might have one task that broadcasts the same data out to all members of a group, or collective, or it scatters different pieces of the data out to each of the members to process.

    • Afterwards, that task can gather all of the individual results from the members of the group and combine them for a final output.

    • When operations require this type of global communication, it's important to consider how it can grow and scale. Simply establishing point-to-point pairs may not be sufficient.

    • If one task is acting as a centralized manager to coordinate operations with a group of distributed workers, as the number of workers grow, the communication workload of the central manager grows too and may turn it into a bottleneck.

    • This is where strategies like divide and conquer can be useful to distribute the computation and communication in a way that reduces the burden on any one task.

    • These are just a few high-level structures to serve as a starting point as you begin to plan the communications for a parallel program.

  • A few other factors to consider include whether the communications will be synchronous or asynchronous.

    • Synchronous communications are sometimes called blocking communications because all tasks involved have to wait until the entire communication process is completed to continue doing other work.

      • That can potentially result in tasks spending a lot of time waiting on communications instead of doing useful work.

    • Asynchronous communications, on the other hand, are often referred to as nonblocking communications because, after a task sends an asynchronous message, it can begin doing other work immediately, regardless of when the receiving task actually gets that message.

  • You should also consider the amount of processing overhead a communications strategy involves, because the computer cycles spent sending and receiving data are cycles not being spent processing it.

  • Latency is another factor to consider, the time it takes for a message to travel from point A to B expressed in units of time like microseconds, and

  • bandwidth, which is the amount of data that can be communicated per unit of time, expressed in some unit of bytes per second.

  • Now, if you're just writing basic multi-threaded or multi-processed programs to run on a desktop computer, some of these factors like latency and bandwidth probably aren't major concerns because everything is running on the same physical system. But as you develop larger programs that distribute their processing across multiple physical systems, those inter-system communication factors can have a significant impact on the overall performance.

PreviousagglomerationNextMapping

Last updated 5 years ago

Was this helpful?