Performance

• Hygiene factor

  • Nobody realises a high performant system, until it is not performing well enough (leads to compliants or user something else)

Perceived Performance

• how quickly a system appears to perform its task.

• people have implicit expectations about performance, they are seldom expressed in hard figures

  • people tend to overestimate their own patience

  • Need to find these numbers, SLAs

• People tend to value predictability in performance

  • When the performance of a system is fluctuating, they remember a bad experience, even if the fluctuation is relatively rare

  • Need to implement consistent perfromance

• If a certain task takes a long time, it helps to inform the user about how long it will take.

  • ie use of progress bars

  • acceptance and wait time increases of users

• when the system seems unresponsive with no apparent reason, people get irritated very quickly.

• Increasing the real performance of a system can increase the perceived performance.

  • But real performance may not be able to increase (high cost/physics)

  • Use of splash screens (ie word) and progress bars

    • provide visual feedback that it is processing request

    • still users processing power

    • Use of ads (ie youtube to load videos)

Performance during infrastructure design

• Without proper design, systems will respond to increased load with some degree of decreasing performance

• Design will help meet performance load increases

• performance must be considered not only when the system works as expected, but also when the system is in a special state, ie:

• Parts have failed

• The system is in maintenance

• A backup is performed

• Batch jobs are running

• Calculating performance in the design phase is extremely difficult and very unreliable

  • Only on very small parts of the system, with predictable load, the performance can be calculated by analyzing each and every step of the implemented process.

  • Doing this is costly

• Ways of calculating performance

  • Benchmarking

  • Using vendor experience

  • Prototyping

  • User Profiling

Benchmarking

• Comparing specs is not great to compare performance, Benchmarking is better

• A benchmark uses a specific test program to assess the relative performance of an infrastructure component.

• used to assess the performance characteristics of computer hardware or software

• Benchmarks provide a method of comparing the performance of various subsystems across different system architectures

• Be aware that benchmark programs heymeasure the raw performance, not taking into account the typical usage of such components.

• benchmarks are only useful for comparing the raw speed of parts of an infrastructure

Using vendor experience

• Best way, let the vendor tell you

  • but must be exeperienced and trusted

• They know their products and gather stats

• they can provide tools, figures, and best practices to help design the appropriate infrastructure for their products.

• Can be used as consultants, but this costs

Prototyping

• Use a prototype at an early stage of implementation

• ie by hiring equipment from suppliers, by using datacenter capacity at a vendor’s premise or by using cloud computing resources.

• To find potential performance issues as early as possible in the design process

• Prototyping should focus on those parts of the system that pose the highest risk

• A proof of concept (POC) should be used to test the most challenging parts of your solution early in the project.

  • Most people deal with easy parts then challenging parts later on

  • but challenging parts could lead to a delay in the project or even a halt.

• POC shows to both the project team and the customer that the project's highest risk has been taken care of

User profiling

• used to predict the load a new software system will pose on the infrastructure, and to be able to create performance test scripts that put representative load in the infrastructure before the software is actually built

• NEed to have a good indication of the expected usage of the system

  • defining a number of typical user groups of the new system (also known as personas)

    • persona groups must be interviewed to understand how they will use the new system. producing a list of main tasks

    • For each of these tasks, estimations can be made on how, and how often they will use the system’s functionality to perform the task.

  • by creating a list of tasks they will perform on the new system.

  • Thus can figure out the amount data used accross network, hard disk written and read

  • This is just an idea, live functioning system the load on the system for a specific task can be very hard to predict and will have:

    • many personas

    • complex tasks

    • tasks are spread in time

    • show hotspots (like starting the application or logging in, which typically happens at the start of the day)

    • High load time (sales)

    • the system can have background processes running

Performance of a running system: Managing bottlenecks

• The performance of a system is based on the performance of all its components, and the interoperability of various components

  • measuring the performance of a system only has value if the complete system is taken into account.

  • ie building an infrastructure with really fast networking components has little benefits when the used hard disks are slow

• A performance problem may be identified by slow or unresponsive systems

  • usually occurs because of high system loads, causing some component of the system to reach some limit.

  • This component is referred to as the bottleneck

  • To find this bottleneck, performance measurements are needed

• Finding the bottleneck allows us to improve performance

• When a bottleneck is removed, usually another bottleneck arises

  • no matter how much performance tuning is done, there will always be a bottleneck

• bottlenecks are ok as long as it does not negatively influence performance of the complete system under the highest expected load

Performance of a running system: Performance testing

• Benchmarking is a way to measure individual components, while system performance tests measure the system as a whole

• Three major types of performance tests

  • Load testing

    • This test shows how a system performs under the expected load.

    • It is a check to see if the system performs well under normal circumstances.

  • Stress testing

    • This test shows how a system reacts when it is under extreme load.

    • Goal is to see at what point the system "breaks" (the breakpoint) and where it breaks (the bottleneck)

  • Endurance testing

    • This test shows how a system behaves when it is used at the expected load for a long period of time

    • finding out when the system will have performance degredation

    • Typical issues:

      • memory leaks

      • expanding database tables

      • filling disks

• How

  • Performance testing software typically uses one or more servers to act as injectors

    • each emulating a number of users, each running a sequence of interactions

  • A separate server acts as a test conductor, coordinating the tasks, gathering metrics from each of the injectors, and collecting performance data for reporting purposes

  • The usual sequence is to ramp up the load, by some factor

  • The test result shows how the performance varies with the load, given as number of users versus response time.

• Performance testing should be done in a production-like environment

  • Done in development environment usually lead to results that are highly unreliable.

  • To reduce cost, sometimes it is advisable to use a temporary test environment

    • with similar hardware as prod

Performance patterns

Increasing performance on upper layers (ie software)

• it is good practice to first check for performance optimizations in the upper layers

• Database and application tuning typically provides much more opportunity for performance increase than installing more computing power

• Application performance can benefit from

  • prioritizing tasks,

  • working from memory as much as possible (as opposed to working with data on disk)

  • making good use of queues and schedulers

• Improving performance of software only works when you have access to source code. For commercial off-the-shelf software, this is usually not feasible.

• Tuning the databases used by the application, by for instance adding indexes, can be an opportunity to significantly improve performance

• understand how applications work on a multithreaded system, and take advantage of multiple cores or multiple instances of applications

Caching

• improves performance by retaining frequently used data in high speed memory, reducing access times to data

• Different sources that provide data are slower than others

  • over the network is slower than ram

• Used for replacing reading from hard disk or from the network

• Disk caching

  • To speed up the reading of data from disk, disk drives contain cache memory

  • This cache memory stores all data recently read from disk, and some of the disk blocks following the recently read disk blocks.

    • When the data is read again, or (more likely) the data of the following disk block is needed, it is fetched from high speed cache memory.

  • implemented in the storage component itself

  • cache used on the physical disks

  • cache implemented in the disk controller)

  • implemented in the operating system

  • The general rule of thumb that adding memory in servers improves performance is due to the fact that all non-used memory in operating systems is used for disk cache

    • Over time, all memory gets filled wit previously stored disk requests and prefetched disk blocks, speeding up applications

Web proxies

• Type of proxy

• instead of fetching all requested data from the internet each time, earlier accessed data can be cached in a proxy server and fetched from there

• benefits:

  • users get their data faster than when it would be retrieved from a distant web server

  • all other users are provided more bandwidth to the internet, as the data did not have to be downloaded again

Operational data store

• read-only replica of a part of a database for a specific use

• Instead of accessing the main database for retrieving information, often used information is retrieved from a separate small ODS database, not degrading the performance of the main database

• Have replica of the data in main db in the ODS

Front-end servers

• In web facing environments storing most accessed (parts of) pages on the web front-end server (like the static pictures used on the landing page) significantly lowers the amount of traffic to back-end systems.

  • CDN

• Reverse-proxies can be used to automatically cache most requested data as well.

In-memory databases

• used in situations where performance is crucial (like in real-time SCADA systems). Of course, special arrangements must be made to ensure data is not lost when a power failure occurs.

Scalability

• indicates the ease in with which a system can be modified, or components can be added, to handle increasing load

• A system whose performance improves after adding hardware, proportionally to the capacity added, is considered to scale well

  • vertical scaling

• two ways to increase the scalability of a system:

  • Vertical scaling (also known as scale up) means adding resources to a single component in a system,

    • typically adding CPUs or memory to a server.

    • here is always a limit to how far a system can be expanded. physics

  • Horizontal scaling (also known as scale out) means adding more components to the infrastructure,

    • such as adding a new web server in a pool of web servers, or adding disks in a storage system.

    • drawback

      • larger numbers of components also mean increased management complexity, as well as a more complex programming model and issues such as throughput and latency between nodes.

      • pays off in the long run, as it is possible to scale up much more since there is a much higher upper limit.

    • works best when the system is partitioned

    • Parts of the system can scale independently of other parts of the system.

    • When more load is placed on the system, somewhere in the system a bottleneck will occur. At that point additional infrastructure components are added to cope with the load.

      • When needed, the system can be expanded further, either in the same tier, or on another tier

    • Doubling the number of components does not necessarily double the performance

      • Because of overhead

        • the extra scheduling needed inmultiprocessor systems, or buffering and link state issues in network connections

      • doubling components usually only provides about 70% to 80% performance increase.

      • Adding more components leads to even more diminishing returns.

Load balancing

• Used to spread the load over various machines, horizontally scaled replicas

• A load balancer automatically redirects tasks to members in the server farm.

• checks the current load on each server in the farm and sends incoming requests to the least busy server.

  • Or using some algorithm

    • number of connections a server has

    • the measured response time of a server

• it increases availability:

  • when a server in the server farm is unavailable, the load balancer notices this and ensures no requests are sent to the unavailable server until it is back online again

• the availability of the load balancer itself becomes very important in this setting and load balancers are typically setup in a failover configuration.

• Load balancers must be identical

• The application running on a load balanced system must be able to cope with the fact that each request can be handled by a different server.

  • The application (or at least the load balanced parts of the application) must be stateless for this to work

High performance clusters

• by combining many computer systems create high performance compute

• Usually a large number of cheap off theshelf servers are used, connected by high speed network

• used for calculation-intensive systems like weather forecasts, geological, nuclear, or pharmaceutical research

• The challenge is to have all systems doing useful calculations all of the time, without wasting too many resources and too much time communicating to other systems in the cluster.

• Many run on some varient of linux

Grid Computing

• is a high performance cluster that consists of systems that are spread geographically.

  • The limited bandwidth is the bottleneck when architecting grid systems.

• Example

  • SETI@HOME

    • These types of grids utilize the unused computer time of PCs (for instance when the computer is displaying its screensaver).

• Secruity is important

  • PCs running calculations must be sufficiently secured against illegal use by third parties.

  • it should not be possible to alter data that is sent through the grid and the grid infrastructure must ensure the PCs calculate their tasks as expected

    • One way of handling this is to have each calculation executed twice, by two different PCs in the grid, and check for mismatches

Design for use

• In general, it must be known what the system will be used for.

  • Different applications need to be built differently

• In some cases, special products must be used for certain systems

  • Real-time operating systems, in-memory databases, or even specially designed file systems can be a solution for special performance sensitive systems.

• Most vendors of databases, web servers, operating systems, and storage or network solutions have standard implementation plans that ar proven in practice.

  • In general, try to follow the vendor's recommended implementation.

  • good idea to have the vendors check the design you created.

• When possible, try to spread the load of the system over the available time.

  • ie Making certain that a backup job is not scheduled when some critical report is compiled is also a good idea

• To increase performance, sometimes it is possible to move rarely used data from the main systems to other systems.

  • Large databases are slower than small ones.

  • Moving old data to a large historical database can speed up a smaller sized database.

Capacity management

• This must be implemented if to guarantee high performance of a system in the long term

• the performance of the system is monitored on a continuous base, to ensure performance stays within acceptable limits. Trend analyses can be used to predict performance degradation before users start to notice it

• regular communication with the business allows systems managers to anticipate on business changes