Learning
  • Software Engineering Golden Treasury
  • Trail Map
  • Caching
    • Alternatives to use before using cache
    • Caching Architecture
    • Cache Invalidation and Eviction
    • Cache Patterns
    • Cache
    • Consistency
    • Distributed Caching
    • Issues with caching
    • Types of caches
  • Career
    • algo types
    • Backend Knowledge
    • Burnout
    • consultancy
    • dev-level
    • Enterprise Developer
    • how-to-get-in-tech-from-other-job
    • how-to-get-into-junior-dev-position
    • induction
    • Interview
    • junior
    • mid
    • New Job
    • paths
    • Principle/staff Engineer
    • Requirements for job
    • Senior Dev capabilities
    • learning
      • automating-beginner
      • company1
        • analyst-progression
        • core-eng-progression
        • dev-progression
        • perf-eng-progression
        • soft-deliv-progression
    • mentoring
      • mentor-resources
    • recruitment
      • questions
      • Spotting posers
  • Computer Science
    • boolean-algebra
    • Compiler
    • Finite State Machine
    • Hashing
    • Algorithms
      • Breadth Firth Search
      • complexity
      • Depth First Search
      • efficiency
      • Sliding Window
      • sorting
    • data-structures
      • AVL Trees
      • data-structures
      • Linked List
    • machines
      • Intel Machine
      • Turing Machine
      • von neumann machine
      • Zeus Machine
  • devops
    • The 5 Ideals
    • microservice
    • Artifact repository
    • Bugs and Fixes
    • Build police
    • cloud-servers
    • Deployments
    • Environments
    • GitOps
    • handling-releases
    • infrastructure-as-code
    • System Migrations
    • SDP
    • On Premises Hosting
    • Properties/configuration
    • Release process
    • Release
    • Roll Outs
    • serverless
    • Serverless
    • Cloud Services
    • Versioning
    • AWS
      • deploy-docker-esc
      • cloud-practitiioner-essentials-notes
        • Module 1 - Intro to AWS
        • Module 2 Compute in the cloud
        • Module 3 Global Infrastructure and Reliability
        • Module 4 Networking
        • Module 5 Storage and Databases
        • Security
        • 7 Monitoring and Aanlytics
        • 8 Pricing and Support
        • 9 Migration and Innovation
      • developer-associate
        • AWS Elastic Beanstalk
    • build-tools
      • Managing dependecies
      • Apache ANT
      • Gradle
        • Custom Plugins
        • local-jars
      • Project Management - maven
        • Archtypes
        • Build Lifecycles
        • Customising build lifecycle
        • Dependencies
        • Directory layout
        • jar-files
        • one-to-one
        • Modules
        • Phases
        • Maven Plugins
        • POM
        • profiles
        • setup
        • Starting a maven project
        • wrapper
    • CI/CD
      • Continuous Delivery
      • zookeeper
      • Continuous Integration (CI)
      • github-actions
      • Pipeline
      • Teamcity
    • Cloud computing
      • Overview
      • Service Models
      • Cloud Services
    • containers
      • Best Practices
      • Docker
    • Infrastructure
      • IT Infrastructure Model
      • Non functional Attributes (Quality Attributes)
        • Infrastructure Availability
        • Performance
        • Secruity
    • monitoring
      • Alerting
      • Monitoring & Metrics
      • Metrics
      • Ready pages
      • Splunk
      • Status pages
      • notes-devops-talk
      • logging
        • logging
        • issues
        • Logging
        • Logging
    • Service mesh
      • Service Discovery
      • Istio
    • Terraform
    • container-management
      • Kubernetes
        • commands-glossary
        • OLTP
        • config-maps
        • Links
        • ingress
        • SDP
        • minikube
        • filter
        • indexes
        • sidecar
        • continuous-deployment
  • General Paradigms
    • CAP theorem
    • designing data-intensive applications summary
    • a-philosophy-of-software-design-notes
    • Aspect oriented Programming (AOP)
    • Best Practice
    • Cargo Cult
    • Clean Code
    • Coding reflections
    • Cognitive Complexity
    • Complexity
    • Conventions
    • Design discussions
    • Design
    • Error Handling Checklist
    • Exceptions
    • Feature Flags/toggle
    • Functional requirements
    • Last Responsible Moment
    • Lock In
    • Named Arguments
    • Naming
    • Performance Fallacy
    • Quality
    • Redesign of a system
    • Resuse vs Decoupling
    • Rules for software designs
    • Sad Paths
    • Scaling Webservices
    • Scientific Method
    • stream-processing
    • Upstream and Downstream
    • Patterns
      • Client-SDK-Pattern
      • ORM
      • Api gateway
      • Business Rules Engine
      • cache
      • Composition Root
      • Dependency Injection Containers
      • Dependency Injections
      • Double Dispatch
      • Exception Handling
      • Gateway pattern
      • Humble Object
      • Inheritance for reuse
      • Null Object Pattern
      • Object Mother
      • Patterns
      • Collection pipeline pattern
      • Service Locator
      • Setter constructor
      • Static factory method
      • Step Builder Pattern
      • telescopic constructors
      • Toggles
      • API
        • Aims of API designs
        • Avoid Checked Exceptions
        • Avoid returning nulls
        • Be defensive with your data
        • convience-methods
        • Fluent Interfaces
        • Loan Pattern
        • prefer-enums-to-boolean-returns
        • return-meaningful-types
        • Small intefaces
        • Support Lambdas
        • Weakest type
      • Gang of Four
        • Builder
        • Factory Pattern
        • Strategy Pattern
        • Template
        • abstract Factory
        • Adapter
        • Bridge Pattern
        • Chain of responsibility
        • Command Pattern
        • Composite Design Pattern
        • Decorator Pattern
        • Facade Pattern
        • Flyweight pattern
        • Guard Clause
        • Interpreter
        • html
        • Mediator Pattern
        • Memento Pattern
        • Observer
        • Prototype
        • Proxy
        • Singleton
        • State Pattern
        • Visitor Pattern
    • Architecture
      • Entity Component System
      • Integration Operation Segregation Principle
      • Adaptable Architecture
      • Architecture
      • C4 Modelling
      • cell-based
      • Clean/Hexagonal Architecture
      • Codifying architecture
      • Correct By configuration
      • Cost Base Architecture
      • Data Oriented Design
      • deliberate
      • Domain oriented DOMA
      • Event Driven Architecture
      • Evolutionary Architecture
      • examples
      • Feature Architecture
      • Framework and Libraries
      • functional-core-imperative-shell
      • Layered Architecture
      • Micro services
      • monoliths-to-services
      • Multi tiered Architecture
      • Multi tenant application
      • Resilient Architecture
      • stage event driven architecture (SEDA)
      • links spring rest app
      • Tomato Architecture
      • Tooling
      • Types of architecture
      • checklist
        • Checklist for new project
        • Back end Architecture Checklist
        • Front end Architecture Checklist
        • Mobile Architecture Checklist
      • Cloud Patterns
        • Command and Query Responsibility Segregation (CQRS)
        • Event Sourcing & CQRS
        • Asynchronous Request and Reply
        • Circuit Breaker
        • Retry
        • Sidecar
        • Strangler pattern
      • Domain driven design
        • value & entity
      • Microservices
        • Alternatives to choosing microservices first when scaling
        • Consistency in distributed systems
        • 12 Factor applications
      • Modularity
        • Module monolith vs Microservices
        • Spring Moduilth
      • Architecture Patterns
        • Hexagonal architecture
        • Inverting dependencies
        • Layering & Dependency Inversion Principle
        • Mappings
        • Vertical Slice architecture
        • Web Client Server
        • domain
          • Business and Data Layers Separation
          • DTO
          • Domain Model Pattern
          • Domain Object
          • Transaction Script/ Use Case pattern
        • Enterprise Patterns
          • Concurrency
          • Distribution strategies
          • Domain layer patterns
          • Layering/organisation of code
          • Mapping to datasource
          • Session State
        • Usecases
          • Use case return types
      • Serverless
        • Knative
    • Design architecture aims
      • back of envelope
      • Design ideas
      • Design mistakes
      • high-volume-design
      • ISO Quality Attributes
      • Non functional requirements
      • “Designing for Performance” by Martin Thompson
      • High Performance
      • Qaulity Attributes
        • Availability
        • System Availability
        • Fault Tolerance
        • interoperability
        • Latency
        • Maintability
        • Modifiability
        • Performance
        • Readability
        • Reliability
        • Scalability vs performance
        • Scalability
        • Scaling
        • statelessness
        • Testability
        • Throughput
      • System Design
      • web-scalability-distributed-arch
        • scalable-and-distributed-web-architecture
    • README
      • Conflict-free Replicated Data Type
      • Fallacies
      • Load balancing
      • Rate Limiting
      • Transactions
    • Patterns of Enterprise Application Architecture
      • Repository Pattern
      • Rules Engines
      • scatter-gather
      • Specification Design Pattern
      • Table Driven Development
      • Workflow Design Patterns
        • Triggers
    • Principles
      • Do It Or Get Bitten In The End
      • Dont Repeat Yourself
      • Habitability
      • Keep it simple
      • Responsibility Driven Design
      • Ya Ain’t Gonna Need It
      • Conceptual Overhead
      • CUPID
      • Reuse existing interfaces
      • Facts and Fallacies
      • locality of behaviour
      • Separation of Concerns
      • Simplicity
      • SLAP principle
      • Step down rule
      • Unix Philosophy
      • Wrong abstractions
      • SOLID
        • 1. Single Responsibility Principle
        • 2. Open Close Principle
        • 3. Liskov Substitution Principle
        • 4. Interface Segregation Principle
        • 5. Dependency Inversion Principle
        • GRASP (General Responsibility Assignment Software Principles)
        • Solid for packages
          • jobs
          • CCP
          • CRP
          • REP
          • egress
          • gossip-protocol
        • STUPID
    • programming-types
      • Coding to Contract/Interface
      • Links
      • Declarative vs Imperative Programming Languages
      • defensive-programming
      • Design by contract
      • Domain Specific Languages (DSL)
      • Event Driven
      • file-transfers
      • Logical Programming
      • Mutability
      • Self Healing
      • Simplicity
      • Type Driven Design
      • Value objects
      • Aspect Oriented Programming
      • Concurrent and Parallel Programming
        • Actor Model
        • Asynchronous and Synchronous Programming
        • Batch processing
        • Concurrency Models
        • SAP
        • Multithreading
        • Non Blocking IO
        • Optimistic vs Pessimistic Concurrency
        • Thread per connection or request model
        • Actor
        • aysnchronous-tasks
          • Computational Graphs
          • Divide and conquer
          • Future
          • Thread Pool
        • barriers
          • Barriers
          • Race conditions
        • design
          • agglomeration
          • Communication
          • Mapping
          • Partitioning
        • Liveness
          • Abandoned Lock
          • Deadlocks
          • Livelock
          • Starvation
        • locks
          • Read write lock
          • Reentrant lock
          • Try Lock
        • Mutual Exclusion
          • Data Races
          • Mutual Exclusion AKA Locks
        • performance
          • Amdahl's Law
          • Latency, throughput & speed
          • Measure Speed up
        • synchronization
          • Condition variable
          • producer consumer pattern
          • Semaphore
        • Threads and processes
          • Concurrent and parallel programming
          • Daemon Thread
          • Execution Scheduling
          • sequential-parallel
          • Thread Lifecycle
          • threads-and-processes
      • Functional Programming
        • Currying
        • design-patterns-to-func
        • imperative-programming
        • First class functions
        • Functional Looping
        • Higher Order Functions
        • Immutability
        • Issues with functional Programming
        • Lambda calculus
        • Lazy & Eager
        • map
        • Monad
        • Railway Programming
        • Recursion
        • Reduce
        • referential-transparacy
        • Referential transparency
        • Supplier
      • oop-design
        • Issues with object oriented code
        • Aggregation
        • Anti Patterns
        • Association
        • class-and-objects
        • Composition
        • general-laws-of-programming
        • general-notes
        • Getters and Setters
        • Inside out programming
        • Inversion of control
        • oop-design
        • Other principles
        • Outside in programming
        • Readability
        • Why OO is bad
        • README
          • abstraction
          • encapsulation
          • inheritance
          • Polymorphism
        • clean-code
          • Code Smells
          • Comments
          • Naming
          • CLEAN design
            • code is assertive
            • Cohesion
            • Connascence
            • Coupling
            • Encapsulation
            • Loose Coupling
            • Nonredundant code
      • Reactive Programming
        • reactive-programming
    • Projects and Software types
      • Applicatoin Development
      • Buying or creating software
      • Console Applications
      • Embedded Software development
      • Enterprise
      • Framework Development
      • Games
      • Library development
      • Rewriting
      • White Label Apps
    • State Machines
      • Spring State Machine
  • Other
    • 10x devs
    • Aim of software
    • Choosing Technologies
    • Coding faster
    • Component ownership
    • developer-pain-points
    • Developer Types
    • Effective Software design
    • Full Stack Developer
    • Good coder
    • Issues with Software Engineering and Engineers
    • Learning
    • Logic
    • Role
    • Software Actions
    • Software craftmanship
    • Software Designed
    • Software Engineering
    • Software
    • article-summaries
      • General notes
      • Summary of The Grug Brained Developer A layman's guide to thinking like the self-aware smol brained
      • improve-backend-engineer
      • Optimising Api
      • Simple and Easy
    • README
  • Hardware
    • Cpu memory
    • Storage
  • Integration
    • GRPC
    • API
    • Apis and communications between apps
    • asynchronous and synchronous communications
    • Batch Processing
    • Communications between apps
    • Delivery
    • Distributed Computing
    • Entry point
    • Event Source
    • SDP
    • egress
    • Graphql
    • Idempotency
    • Libraries
    • Long Polling
    • Multiplexing & Demultiplexing
    • Publish Subscribe
    • Push
    • Request & Response
    • REST
    • Remote Method Invocation
    • Remote Procedure Calls
    • Server Sent Events
    • Short Polling
    • Sidecars
    • SOAP
    • Stateless and Stateful
    • Streams
    • Third Party Integrations
    • wdsl
    • Web Services
    • Webhooks
    • repository
    • Kafka
      • Kafka Streams
    • message-queues
      • ActiveMQ
      • Dead Letter Queue
      • JMS
      • Messaging
  • Languages
    • C
    • Choosing A Language
    • cobol
    • Composite Data Types
    • creating
    • Date time
    • Numbers
    • Pass by value vs Pass by reference
    • Primitive Data Types
    • REST anti-patterns
    • Rust
    • Scripting
    • Static typing
    • string
    • Task Oriented Language
    • assembly
    • Getting started
      • Functional Concepts
    • cpp
    • Java
      • Code style
      • Garbage Collection
      • Intellij Debugging
      • Artifacts, Jars
      • Java internals
      • Java resources
      • Java versions
      • JShell
      • Libraries
      • opinionated-guide
      • Starting java
      • Java Tools
      • Why use java
      • Advanced Java
        • Annotations
        • API
        • Database and java
        • Debugging Performance
        • Files IO
        • Finalize
        • JDBC
        • jni
        • Libraries
        • Logging
        • SAP
        • Memory Management
        • Modules
        • OTher
        • Packaging Application
        • Pattern matching
        • performance
        • Properties
        • Reference
        • reflection
        • Scaling
        • Scheduling
        • secruity
        • Serilization
        • Time in Java
        • validation
        • Vector
        • Concurrency and Multithreaading
          • Akka
          • ExecutorCompletionService
          • Asynchronous Programming
          • Concurrency and Threads
          • CountDownLatch
          • Conccurrent Data Structures
          • Executor Service
          • Futures
          • reactive
          • Semaphore
          • structured concurrency
          • Threadlocal
          • Threads
          • Virtual Threads
          • Mutual Exclusion
            • Atomic
            • Synchronized
            • Thread safe class
            • Threads
        • debug
          • heap-dumps
          • thread-dumps
        • functional
          • Collectors
          • Exception Handling
          • Flatmap
          • Functional Programming
          • Generators
          • Immutability
          • issues
          • Optional
          • Parallel Streams
          • Reduce
        • networks
          • HTTP client
          • servlet-webcontainers
          • sockets
          • ssl-tls-https
      • Basics of java
        • compilation
        • computation
        • Conditonal/Flow control
        • Excuting code
        • Instructions
        • Looping/Iterating
        • memory-types-variables
        • methods
        • Printing to screen/debugging
        • Setup the system
        • Data structures
          • Arrays
          • Arayslist/list
          • Map
      • Effective Java notes
        • Creating and Destroying Objects
        • Methods Common to All Objects
        • best-practice-api
        • Classes and Interfaces
        • Enums and Annotations
        • Generics
      • framework
        • aop
        • bad
        • Dagger
        • Databases
        • Lombok
        • Mapstruct
        • netty
        • resliance4j
        • RxJava
        • Vert.x
        • Spring
          • Spring Data Repositories
          • actuator
          • cloud-native
          • H2 Db in Spring
          • Initializrs
          • JDBC Template
          • Java Persistence API (JPA)
          • kotlin
          • Pitfalls and advice
          • PRoxies
          • Reactive
          • spring security
          • spring-aop
          • Spring Boot
          • spring-jdbc
          • Spring MVC
          • Spring Testing
          • Testing
          • Transaction
          • patterns
            • Component Scan Patterns
            • Concurrency
            • Decorator Pattern in Spring
        • Micronaut
          • DI
        • Quarkus
          • database
          • Links
      • Intermediate level java
        • String Class
        • Assertions
        • Casting
        • Clonable
        • Command line arguments
        • Common Libraries/classes
        • Comparators
        • Where to store them?
        • Shallow and Deep Copy
        • Date and Time
        • Enums
        • Equals and Hashcode
        • Equals and hashcode
        • Exceptions
        • Final
        • Finally
        • Generics
        • incrementors
        • Null
        • packages and imports
        • Random numbers
        • Regex
        • Static
        • toString()
        • OOP
          • Accessors
          • Classes
          • Object Oriented Programming
          • Constructors
          • Fields/state
          • Inheritence
          • Interfaces
          • Methods/behaviour
          • Nested Classes
          • Objects
          • Static VS Instance
          • Whether to use a dependency or static method?
        • Other Collections
          • Other Collections
          • Arraylist vs Linkedlist
          • LinkedHashMap
          • Linked List
          • Priority queue
          • Sequenced Collections
          • Set
          • Shallow vs Deep Copy
          • Time Complexity of Collections
          • What Collection To use?
    • kotlin
      • Domain Specific Language
      • learning
      • Libraries
      • Personal Roadmap
      • Links
    • Nodejs
      • Performance
  • Management & Workflow
    • Agile
    • Take Breaks
    • # Communication
    • Engineering Daybook
    • Estimates
    • Feedback Loops
    • Little's law
    • Managing Others
    • poser.
    • Presentations
    • self-improvement
    • software-teams
    • Task List
    • trade-off
    • Types of devs
    • Type of work
    • Waterfall Methodology
    • coding-process
      • Bugs
      • Code Review
      • Code Reviews
      • Documentation
      • Done
      • Handover
      • Mob Programming
      • Navigate codebase
      • Pair Programming
      • Pull Requests
      • How to do a story
      • Story to code
      • Trunk based development
      • Xtreme Programming (XP)
      • debugging
        • 9 Rules of Thumb of Dubugging
        • Debugging
        • using-debugger
      • Legacy code
        • Legacy crisis
        • Working with legacy code
    • Managing work
      • Theory of constraints
      • Distributed Teams
      • estimations
      • Improving team's output
      • Kanban
      • Kick offs
      • Retrospectives
      • Scrum
      • Sign offs
      • Stand ups
      • Time bombs
      • Project management triangle
    • Notion
    • recruitment
      • In Person Test
      • Interviews
      • Unattended test
  • Networks
    • Content Delivery Network - CDN
    • DNS
    • cache control
    • Cookies and Sessions
    • Docker Networking
    • Duplex
    • Etags
    • HTTP Cache
    • HTTP - Hyper Text Transfer Protocol
    • HTTP/2
    • Http 3
    • Internet & Web
    • iptables
    • Keep alive
    • Leader Election
    • Load balancer
    • long-polling
    • Network Access Control
    • Network Address Translation (NAT)
    • Network Layers
    • Nginx
    • OSI network model
    • Persistent Connection
    • Polling
    • Proxy
    • Quic
    • reverse-proxy
    • servers
    • Server sent events (SSE)
    • SSH
    • Streaming
    • Timeouts
    • Url Encoding
    • Web sockets
    • WebRTC (Web Real-Time Communication)
    • Wireshark
    • tcp/ip
      • Congestion
      • IP - Internet Protocol
      • TCP - Transmission Control Protocol
  • Operating Systems
    • Cloud Computing
    • Distributed File Systems
    • Distributed Shared Memory
    • Input/Output Management
    • Inter-Process Communication
    • Threads and Concurrency
    • Virtualization
    • Searching using CLI
    • Bash and scripting
    • Booting of linux
    • makefile
    • Memory Management
    • Processes and Process Management
    • Scheduling
    • Scripting
    • Links
    • Ubuntu
    • Unix File System
    • User groups
    • Linux
  • Other Topics
    • Finite state machine
    • Floating point
    • Googling
    • Setup
    • Unicode
    • Machine Learning
      • Artificial Intelligence
      • Jupyter Notebook
    • Blockchain
    • Front End
      • Single Page App
      • cqrs
      • css
      • Debounce
      • Dom, Virtual Dom
      • ADP
      • htmx
      • Island Architecture
      • Why use?
      • Java and front end tech
      • mermaidjs
      • Next JS
      • javascript
        • Debounce
        • design
        • Event loop
        • testing
        • Typescript
        • react
          • Design
          • learning
          • performance
          • React JS
          • testing
      • performance
      • Static website
    • jobs
      • Tooling
      • bash text editor - vim
      • VS code
      • scaling
        • AI Assistant
        • Debugging
        • General features and tips and tricks
        • IDE - Intellij
        • Plugins
        • Spring usage
  • persistance
    • ACID - Atomicity, Consistency, Isolation, Durability
    • BASE - Basic Availability, Soft state, Eventual Consistency
    • Buffer
    • Connection pooling
    • service
    • Database Migrations - flywaydb
    • Databases
    • Eventual Consistency
    • GraphQL
    • IDs
    • indexing
    • MongoDB
    • Normalisation
    • ORacle sql
    • Partitioning
    • patterns
    • PL SQL
    • Replication and Sharding
    • Repository pattern
    • Sharding
    • Snapshot
    • Strong Consistency
    • links
    • Files
      • Areas to think of
    • hibernate
      • ORM-hibernate
    • Indexes
      • Elastisearch
    • relationships
      • many-to-many
      • SDP
      • serverless
      • x-to-x-relationships
    • sql
      • Group by
      • indexes
      • Joins
      • Common mistakes
      • operators
      • performance
    • types
      • maven-commands-on-intellij
      • in-memory-database-h2
      • Key value database/store
      • Mongo DB
      • NoSQL Databases
      • Relational Database
      • Relational Vs Document Databases
  • Security
    • OAuth
    • API Keys
    • Certificates and JKS
    • Cluster Secruity
    • Communication Between Two Applications via TLS
    • Cookies & Sessions
    • CORS - Cross-Origin Resource Sharing
    • csrf
    • Encryption and Decryption
    • Endpoint Protection
    • JWT
    • language-specific
    • OpenID
    • OWASP
    • Secrets
    • Secruity
    • Servlet authentication and Authorization
    • vault
  • Testing, Maintainablity & Debugging
    • Service-virtualization and api mocking
    • a-test-bk
    • Build Monitor
    • Builds
    • Code coverage
    • consumer-driven contract testing
    • Fixity
    • Living Documentation
    • Mocks, Stubs & Doubles
    • patterns
    • Quality Engineering
    • Reading and working with legacy code
    • Reading
    • remote-debug-intellij
    • simulator
    • Technical Debt
    • Technical Waste
    • Test cases
    • Test Data Builders
    • Test Pyramids
    • Test Types
    • Testing Good Practice
    • Testing
    • What to prime
    • What to test
    • Debugging
      • Debugging in kubernetes or Docker
    • fixing
      • How to Deal with I/O Expense
      • How to Manage Memory
      • How to Optimize Loops
      • How to Fix Performance Problems
    • Legacy Code
      • Learning
      • Legacy code
      • techniques
    • libraries
      • assertj
      • Data Faker
      • Junit
      • mockito
      • Test Containers
      • Wiremock
      • Yatspec
    • Refactoring
      • Code Smells
      • refactoring-types
      • Refactoring
      • Technical Debt
      • pyramid-of-refactoring
        • Pyramid of Refactoring
    • Test first strategies
      • Acceptance Testing Driven Developement (ATDD)
      • Behaviour Driven Development/Design - BDD
      • Inside out
      • Outside in
      • Test driven development (TDD)
    • testing
      • Acceptance tests
      • How Much Testing is Enough?
      • Approval Testing
      • Bad Testing
      • End to end tests
      • Honeycomb
      • Testing Microservices
      • Mutation testing
      • Property based testing
      • Smoke Testing
      • social-unit-tests
      • solitary-unit-tests
      • Static Analysis Test
      • Unit testing
  • Version Control - Git
    • Branch by Abstraction
    • feature-branching
    • Git patches
    • Trunk Based Development
Powered by GitBook
On this page
  • What
  • Key features
  • Where used
  • Disadvantages
  • Scaling
  • When not to use
  • Alternatives
  • How to implement
  • Links
  • Examples Links

Was this helpful?

  1. Networks

Web sockets

PreviousUrl EncodingNextWebRTC (Web Real-Time Communication)

Last updated 1 year ago

Was this helpful?

What

  • a communication protocol that provides full-duplex communication channels over a single TCP connection

  • Defined by the WebSocket Protocol. It is a standardized protocol that defines the rules and format for establishing and maintaining a WebSocket connection between a client and a server.

    • RFC 6455 https://www.rfc-editor.org/rfc/rfc6455

    • operates at the application layer of the TCP/IP stack and is independent of the underlying transport protocol.

    • It uses the standard WebSocket URI scheme, which is

      • "ws://" for unencrypted connections and

      • "wss://" for encrypted connections (WebSocket Secure)

    • Contains two main parts

      • Handshake:

        • the initial step to establish a WebSocket connection.

        • It follows the HTTP protocol to upgrade the connection from HTTP to the WebSocket Protocol.

        • The client sends a special HTTP request to the server, known as the WebSocket handshake request, indicating its desire to switch protocols.

        • The server responds with an HTTP 101 status code (Switching Protocols) if it supports WebSockets, indicating a successful upgrade to the WebSocket Protocol.

      • Data Framing:

        • Once the WebSocket connection is established, data is exchanged in the form of messages between the client and the server.

        • The WebSocket Protocol defines a framing mechanism to encapsulate and format these messages for transmission.

        • The framing includes a header that specifies the message type (text or binary) and the length of the payload.

    • Other features include

      • message fragmentation

      • ping-pong heartbeats for connection monitoring

      • extensions for additional functionality like compression or security

  • supported by most modern web browsers

Key features

  • Persistent Connection:

    • maintain a long-lived connection between the client and the server, allowing for real-time communication.

  • Full-Duplex Communication

    • enable simultaneous communication in both directions.

    • The client can send messages to the server, and the server can push data to the client without waiting for a request.

  • Lightweight Overhead

    • a smaller overhead compared to traditional HTTP connections.

    • The initial handshake is slightly larger, but subsequent messages have minimal framing overhead.

Where used

  • Chat Applications

    • ideal for building chat applications where multiple users need to exchange messages in real-time.

    • With WebSockets, messages can be instantly sent and received, enabling seamless communication between users.

  • Real-Time Collaboration

    • WebSockets are well-suited for collaborative applications such as

      • collaborative document editing

      • project management tools

      • whiteboard applications

    • WebSockets enable real-time updates and synchronization of changes across multiple clients.

  • Live Dashboards

    • display real-time data and updates, such as

      • monitoring systems

      • financial tickers

      • analytics dashboards

    • The server can push updates to connected clients without clients needing to constantly poll the server.

  • Multiplayer Games

    • allowing players to interact with each other in real-time

    • enable quick and efficient communication between players and the game server, facilitating real-time gameplay and synchronization.

  • Stock Market or Auction Applications

    • provide real-time updates and notifications in stock market applications or online auction platforms.

    • Users can receive instant updates on stock prices, bids, or auction status changes without manually refreshing the page.

  • Notification Systems

    • where clients need to be notified instantly about new events, messages, or updates

    • social networks, messaging apps, or any application that requires real-time notifications

  • IoT (Internet of Things)

    • establish a real-time connection between IoT devices and a central server

    • enables devices to send sensor data, receive commands, and provide real-time updates to control systems or user interfaces.

Disadvantages

  • Complexity

    • can be more complex compared to traditional HTTP connections. WebSocket implementation requires additional server-side and client-side code, as well as handling various events and message protocols.

    • This complexity can increase development time and may require a deeper understanding of the WebSocket protocol.

  • Server Resources

    • maintain a persistent connection between the client and the server, which can consume server resources.

    • Each open WebSocket connection requires a certain amount of memory and processing power on the server side. Handling a large number of concurrent WebSocket connections can put a strain on server resources and require careful management to avoid performance issues.

  • Proxy and Firewall Compatibility

    • Some proxies and firewalls are not WebSocket-aware or may not support the WebSocket protocol.

    • This can cause connectivity issues if WebSocket traffic is blocked or improperly handled by intermediate network devices.

    • To ensure WebSocket connectivity, it may be necessary to configure or bypass certain network infrastructure components.

  • Limited Browser Support

    • While modern web browsers widely support WebSockets, some older or less common browsers may lack WebSocket support or have incomplete implementations. If you need to support a wide range of browsers, you may need to consider fallback mechanisms or alternative communication methods for clients that don't support WebSockets.

  • Scalability

    • Scaling WebSocket applications to handle a large number of concurrent connections can be challenging.

    • As the number of active WebSocket connections increases, the server needs to efficiently manage resources and handle the increased load.

    • Load balancing and horizontal scaling strategies may be required to handle high traffic volumes.

  • Stateful Connections

    • WebSockets maintain a stateful connection between the client and the server.

    • This means that the server needs to keep track of the connection state and associated data for each WebSocket client.

    • Managing and synchronizing this state can introduce complexities, especially in distributed or clustered server environments.

Scaling

  • involves ensuring that your application can handle a large number of concurrent connections and deliver real-time updates efficiently.

  • Optimize resource utilization: Make sure your server efficiently manages resources (such as memory and CPU) for WebSocket connections.

  • Consider using lightweight data structures, connection pooling, and efficient event-driven programming models.

  • Set appropriate limits on the number of concurrent WebSocket connections your server can handle.

    • prevent resource exhaustion and ensures fair resource allocation.

  • Distribute incoming WebSocket connections across multiple servers using a load balancer.

    • This enables horizontal scaling and distributes the load evenly among backend servers

  • Vertical scaling: Upgrade your server hardware by adding more powerful CPUs, increasing memory, or utilizing faster network interfaces

  • Horizontal scaling:

    • Add more servers to your WebSocket application by using techniques like server clusters or distributed systems.

    • Each server handles a subset of WebSocket connections, and load balancing distributes connections across multiple servers.

  • Caching for frequently accessed data

  • Pub/Sub Systems

  • Optimize message processing:

    • Minimize the processing required for each message by offloading heavy computations or non-real-time tasks to background processes or worker threads.

    • Prioritize message processing based on importance or timestamp to ensure timely delivery of critical updates

    • Compress WebSocket messages to reduce network bandwidth usage, especially for large payloads.

      • Consider optimizing the size of transmitted data to minimize latency and improve overall performance.

  • Implement monitoring tools and collect metrics to understand the performance and health of your WebSocket infrastructure.

    • Monitor connection counts, server load, latency, and other relevant metrics to identify bottlenecks and make data-driven scaling decisions.

  • Utilize auto-scaling mechanisms provided by cloud platforms to automatically scale your WebSocket infrastructure based on predefined thresholds or metrics to handle varying loads

When not to use

  • application primarily involves simple request-response interactions without the need for real-time updates or continuous communication, using regular HTTP requests may be sufficient

  • Low Latency, High Frequency Data Transfer

    • more specialized protocols or technologies, such as UDP-based solutions or custom networking frameworks should be used

  • Limited Server Resources

    • maintain open connections between clients and servers, which can consume server resources

    • alternatives like long polling or server-sent events that allow for more efficient resource utilization

  • Compatibility Requirements

    • Older technologies may not be able to use this protocol

  • can introduce security considerations, particularly when it comes to cross-origin resource sharing (CORS) and authentication

Alternatives

  • Long Polling

    • a technique where the client sends a request to the server and keeps the connection open until the server has new data to send or a timeout occurs.

    • When new data is available, the server responds to the client, and the client immediately sends a new request to maintain the connection.

    • Long polling can simulate real-time communication effectively but requires more server resources and incurs additional latency compared to WebSockets

  • Server-Sent Events (SSE)

    • unidirectional communication protocol that allows the server to push data to the client over a single HTTP connection.

    • provides a simple API for the server to send data as events, which the client can receive and process.

    • suitable for scenarios where only server-to-client communication is required, without the need for bidirectional communication.

  • Comet is a technique that enables server-to-client pushing of data over HTTP.

    • uses techniques like long polling, hidden iframes, or streaming to establish a persistent connection between the client and server.

    • Comet is a generic term that encompasses various approaches to achieving real-time communication over HTTP, including long polling and streaming.

  • WebRTC (Web Real-Time Communication)

    • set of APIs and protocols that enable real-time communication between web browsers

    • provides peer-to-peer audio, video, and data streaming capabilities without the need for server intermediation

  • GraphQL Subscriptions

    • allows clients to subscribe to specific data changes on the server.

    • the server can push updates to clients when the subscribed data changes. While GraphQL subscriptions are primarily focused on data updates, they can be used for real-time communication in certain scenarios.

How to implement

  • Two sides server and client implementations

  • Server-Side Implementation

    • Genearlly use a library/framework to handle this

    • Set up a WebSocket server

      • creating a WebSocket server object, specifying the desired routes or endpoints

      • defining the logic for handling incoming WebSocket connections and messages.

    • Implement the logic to handle WebSocket events (Event handlers) such as

      • connection establishment

      • message reception

      • errors

      • disconnection

    • Implement the functionality to broadcast messages to all connected clients or send targeted messages to specific clients.

      • This involves keeping track of connected WebSocket clients and sending messages to the appropriate recipients.

  • Client side implementation

    • Create a WebSocket object in your client-side code and establish a connection to the WebSocket server.

      • Provide the appropriate WebSocket server URL or endpoint.

    • Define event handlers for WebSocket events such as connection open, message received, error, and connection close.

      • These event handlers will handle the server's responses and allow your client-side code to react accordingly.

    • Implement the code to send messages from the client to the server using the WebSocket object's send() method. Same for receive

    • Implement the functionality to close the WebSocket connection when it's no longer needed

Links

  • https://www.youtube.com/watch?v=2Nt-ZrNP22A

  • https://www.html5rocks.com/tutorials/websockets/basics/

  • http://cometdaily.com/2008/07/04/html5-websocket/

  • https://ably.com/topic/websockets

  • https://ably.com/blog/websockets-vs-long-polling WebSockets are a thin transport layer built on top of a device’s TCP/IP stack. The intent is to provide what is essentially an as-close-to-raw-as-possible TCP communication layer to web application developers while adding a few abstractions to eliminate certain friction that would otherwise exist concerning the way the web works

  • https://www.youtube.com/watch?v=8ARodQ4Wlf4

  • https://www.youtube.com/watch?v=2Nt-ZrNP22A

Examples Links

  • https://www.youtube.com/watch?v=U4lqTmFmbAM java spring

Web sockets
What
Key features
Where used
Disadvantages
Scaling
When not to use
Alternatives
How to implement
Links
Examples Links