Essential Cybersecurity Knowledge¶

Introduction¶

Cyber security may be understood as a collection of defensive strategies, technological solutions, and operational practices that are intentionally crafted to safeguard devices linked to the internet, with particular emphasis on the software operating on those devices. Traditional methods employed in cyber defense include firewalls, authentication mechanisms, and network-based software systems that observe, trace, and prevent viruses along with other forms of harmful cyber intrusions. These conventional techniques establish a defensive barrier around critical infrastructure. Nevertheless, security threats frequently originate from weaknesses present within applications, and these weaknesses themselves often stem from errors or oversights in the design and execution of both software and network systems.

Solutions have been, and continue to be, created to shield systems against such threats; however, malicious actors persistently take advantage of freshly identified weaknesses. The environment of cyber attacks is in a state of perpetual change, constantly generating novel forms of danger. Consequently, constructing defenses solely based on previously identified attack patterns will inevitably prove inadequate. It is also essential to adopt innovative approaches capable of uncovering hidden or dormant intrusions, thereby enabling the deployment of a more dependable and robust security framework. In this context, machine learning and data mining techniques are increasingly recognized as vital components shaping the future trajectory of cyber security efforts.

It is reasonable to assert that software is ubiquitous in modern society, and ensuring its security is of paramount importance. Achieving robust security can be challenging, yet numerous strategies and frameworks for protecting systems are currently available. Despite their differences, all these approaches aim to address a shared core set of foundational inquiries.

• What exactly is the object of protection?
• Do any recognized threats or vulnerabilities currently exist?
• What consequences would the organization face should its data be compromised, lost, or exposed?
• How valuable is the data to the organization in question?
• What measures can be implemented to reduce or manage the associated risks?

Within the specialized vocabulary of cyber security, the terms asset, threat, vulnerability, and exploit appear most frequently. Depending on the source material, certain of these concepts may overlap in meaning or occasionally be used interchangeably.

• An asset refers to whatever is being safeguarded—something that holds worth for its proprietor. This value might be concrete and physical, such as gold bars or an operational server, or it might be abstract, such as sensitive information or intellectual property.
• threat represents a deliberate intention to inflict harm. Within the realm of cyber security, this can be described as a hostile action directed by an adversary toward a valued asset. Even in cases where the attacker does not intend to cause direct damage, the presence of a threat remains significant. The individual or group responsible for posing such a threat is typically referred to as a threat actor.
• A vulnerability denotes a flaw or weakness within the targeted system. This imperfection could manifest as an error in the application's source code, or as a deficiency in the overall architectural design of the system. Vulnerabilities may also arise as a result of incorrect system configuration settings or actions taken by end users.
• An exploit constitutes a method or technique used to leverage a known vulnerability. The primary goal of such an action is typically to gain unauthorized control or access over the targeted asset. (Social engineering, which is often regarded as a relatively straightforward form of deception or manipulation, represents one distinct category of exploit.)

Cyber security in the real world¶

Why should we care?¶

Software is genuinely present in virtually every aspect of modern life. In the earliest days of computing, software primarily operated on large mainframe systems, and only a very limited number of individuals had any direct interaction with it. Somewhat later in technological history, the personal computer emerged, and owing to its widespread adoption and popularity, software gradually made its way into our private residences. (It is worth noting, however, that even within the home environment, software access remained somewhat constrained, and any potential attacker would still have required physical proximity to the targeted machine in order to carry out an intrusion.)

Eventually, the Internet came into existence and began linking personal computers together on a global scale. While the Internet has proven to be immensely beneficial for legitimate, non-malicious users, it simultaneously simplified the task for attackers, as they no longer needed to be physically present at the location of their intended targets. The progression of computing technology continued unabated, and at a later stage, cellular phones were introduced to the market, eventually evolving into smartphones—devices that are essentially compact, portable computers equipped with wireless connectivity capabilities.

At present, we stand on the threshold of the Internet of Things era, which promises to interconnect virtually every conceivable device to the global network. As illustrative examples, we now encounter smart televisions, intelligent door locks, connected automobiles, and even internet-enabled toasters. The software involved in these systems spans a wide spectrum in terms of scale, ranging from substantial components such as full operating systems down to relatively minor elements like USB device drivers.

We are currently witnessing a continuous and dramatic surge in the quantity of devices that require software to function. The total number of devices currently linked to the Internet is measured in the tens of billions, a figure that is largely attributable to the expansion of the Internet of Things, wherein specialized, single-purpose devices (as opposed to general-purpose computing machines) are being connected directly to the online world. The count of such connected devices continues to climb steadily. This trend generates an escalating demand for software engineers and code developers, who will consequently experience considerable pressure to produce code that is both functionally effective and securely designed, all within constrained timeframes. Developers may frequently encounter unrealistic deadlines that compel them to expedite code deployment into production environments.

In addition to the expanding requirement for greater volumes of code, the overall codebase has also grown substantially more intricate. Every machine is now interconnected, and larger-scale systems are increasingly distributed across multiple locations. Distinct components of these systems may be developed by separate vendors, yet they must still function together seamlessly. Furthermore, the majority of contemporary systems are built upon software frameworks that significantly expand the code footprint even for relatively small applications, and/or are designed to be extended through the use of plugins and add-ons.

When we consider the combined effects of ubiquitous connectivity, heightened system complexity, and extensive extensibility within modern software, we arrive at a notably expansive attack surface that is readily available to potential adversaries. Given this reality, it is perhaps unsurprising that the total count of identified software vulnerabilities has demonstrated a consistent upward trajectory over the course of recent years. Information regarding vulnerabilities, along with associated statistical data, can be accessed and queried through resources such as the National Vulnerability Database and the Common Vulnerabilities and Exposures database (commonly abbreviated as NVD and CVE, respectively).

From the perspective of attackers, the act of launching an assault will continue to remain comparatively easier than the task of mounting a comprehensive defense, for as long as the attacker retains the freedom to strike at any point while defenders are obligated to protect every possible entry point. Nevertheless, through the application of thorough and effective security analysis, many of the more apparent vulnerabilities—in other words, the so-called "low-hanging fruit"—can be successfully identified and addressed. This process does not necessarily render a system completely impervious to attack, but it does enhance the system's capacity to resist intrusion attempts perpetrated by less skilled adversaries or automated attack tools.

Discovering vulnerabilities remains a challenging endeavor. Even in cases where the software itself, or the communication protocol it implements, has reached a mature and well-established stage of development, it may still harbor unexpected weaknesses. Below are several illustrative examples of relatively "recent" vulnerabilities that were uncovered in widely recognized and heavily utilized software components:

• (Beast)[https://blog.qualys.com/ssllabs/2013/09/10/is-beast-still-a-threat]
• (Heartbleed)[https://heartbleed.com/]
• (Heist)[http://arstechnica.com/security/2016/08/new-attack-steals-ssns-e-mail-addresses-and-more-from-https-pages/]
• (Krack)[https://arstechnica.com/information-technology/2017/10/severe-flaw-in-wpa2-protocol-leaves-wi-fi-traffic-open-to-eavesdropping/]

The world is undergoing constant transformation, and cyber security has emerged as a rapidly expanding issue of global significance. Adversaries manifest in a wide variety of forms and levels of sophistication, ranging from inexperienced "script kiddies" to state-sponsored actors representing foreign governments, and including organized criminal enterprises. Irrespective of the specific nature of the adversary, all such parties now enjoy relatively straightforward access to highly advanced and potent technological tools. Moreover, certain attack methods are so covert and subtle that they only become apparent after the malicious activity has already been completed.

Successful attacks can generate considerable chaos and inflict substantial harm, often resulting in significant financial losses; however, accurately quantifying the broader business impact of a security breach can prove quite challenging. This particular domain represents an area where it is difficult to obtain definitive, comprehensive, or universally representative statistics or conclusions. At least one notable study, however—conducted by Oxford Economics (the PDF version of which can be accessed via the Wayback Machine)—offers a collection of findings derived from surveys and detailed case analyses. These findings demonstrate that cyber attacks do, in fact, produce major and measurable business consequences for the organizations that fall victim to them.

The primary responsibility of a vulnerability researcher involves formulating practical recommendations aimed at reducing the level of risk faced by an organization. The broader tasks associated with safeguarding enterprise-level systems and sensitive data encompass the establishment of appropriate policies, operational practices, and technical tools that collectively serve to diminish the likelihood of unauthorized or illicit activities. The technical security evaluations carried out by vulnerability researchers typically include the systematic identification of software vulnerabilities, configuration errors, and structural weaknesses within systems.

Media reports only the tip of the iceberg¶

News coverage related to security incidents has become increasingly frequent and prominent in recent times. Reports concerning data breaches are no longer minor stories relegated to the back pages of miscellaneous news sections; instead, they now frequently appear as headline-grabbing, front-page material. Virtually every category of connected device—from infant monitoring systems to home security cameras, from automobiles to high-end luxury yachts—has been targeted and successfully compromised by malicious actors. The types of threats discussed by news organizations have grown progressively more sophisticated and professionally executed. Lists of stolen user credentials have become highly sought-after commodities on underground markets, and reports detailing targeted ransomware campaigns have appeared at rates never before witnessed. This situation reveals an unsettling reality: our digital lives, and by extension our physical, offline existence as well, have become convenient avenues for criminals to generate illicit income, and this pervasive threat impacts virtually every industry sector, geographic region, and social domain.

Although mainstream media outlets are beginning to treat news of security breaches with greater seriousness, and although studies such as the Crime Survey of England and Wales, published by the Office for National Statistics, indicate that the frequency of such incidents is on the rise, it remains a commonly held belief that cyber crime represents a significantly underreported category of unlawful activity.

The information presented above paints a rather sobering and concerning picture of the current state of cyber security affairs. At the same time, however, it is important to acknowledge that governments, private organizations, large corporations, and academic institutions are increasingly channeling greater amounts of financial resources, research efforts, and dedicated personnel toward the goal of improving the overall situation.

Responsibilities and liabilities¶

This particular course will not explore legal matters in extensive depth; however, it is important to note that companies and corporate entities bear certain responsibilities and potential liabilities concerning the data they collect and store. The majority of such organizations have made formal commitments—whether these commitments are legally binding or not—to handle gathered data with the highest degree of care and diligence. In addition, various laws and regulatory frameworks govern the manner in which these entities are required to secure their data holdings and specify the appropriate procedures that must be followed in the event that a security breach occurs.

As one prominent example, the General Data Protection Regulation (GDPR) (officially designated as Regulation (EU) 2016/679) serves to unify and streamline the regulatory requirements pertaining to data protection across all member states of the European Union. The GDPR supersedes the previous directive that governed this area, and its principal objective is to empower individuals with greater control over their personal information. Furthermore, the regulation addresses additional matters such as the transfer of personal data to destinations outside the EU, the imposition of penalties on parties that fail to comply with its provisions, the so-called "right to erasure" (also known as the right to be forgotten), and the mandatory procedures for reporting data breaches. This regulation officially entered into force on the 25^th of May, 2018.

The GDPR establishes the first significant economic rationale for robust data protection practices: in much the same way that environmental regulations have effectively internalized the costs associated with environmental damage into production processes, we may observe a similar dynamic emerging in the realm of data protection. Insurance providers have also taken note of the impending implications of the GDPR and have subsequently introduced various forms of cyber security insurance policies designed to cover certain expenses that may arise as a consequence of a data breach (for instance, compensation for lost revenue or business interruption).

Nevertheless, it is important to recognize that legal frameworks primarily influence those who choose to abide by the law; for individuals engaged in criminal activity, such laws serve merely as a potential deterrent. The Internet affords a significant degree of anonymity and creates physical distance between the attacker and the victim, thereby lowering the barriers to entry for those who might consider stepping into the world of cyber crime. The commission of unlawful acts via the Internet is commonly referred to as hacking. (When discussing the concept of hacking, it is also necessary to mention the practice of ethical hacking. An ethical hacker, much like their criminal counterpart, is a skilled professional who attempts to penetrate computer systems; however, the ethical hacker conducts such activities with explicit authorization from the system owner, with the specific purpose of uncovering security vulnerabilities that malicious actors might otherwise exploit.)

!!! note title="TL;DR Breaking into computers is illegal" The information provided herein is intended strictly for educational purposes and is not meant to facilitate or encourage malicious or damaging cyber attacks. Once again, it must be emphasized that conducting any form of penetration testing requires explicit, documented permission from the legitimate owner of the target system. Moreover, it should be kept in mind that the intended target may be hosted on the same physical server as other unrelated services; consequently, obtaining authorization from the relevant service provider may also be necessary before proceeding with any testing activities.

How the Internet Works¶

The Internet is like an onion¶

The Internet operates through a structured hierarchy composed of multiple layers. The various protocols that facilitate communication across networks can be categorized into four distinct layers.

1. Link layer
2. Internet/network layer
3. Transport layer
4. Application layer

Protocols functioning at the link layer handle direct communication between two entities that share the same physical or logical connection. For instance, a protocol managing communication between a laptop and a WiFi router determines the operating frequency and defines the methods by which individual bits are transmitted across that frequency. At the opposite end of the spectrum, protocols operating at the application layer establish a shared language that enables two applications to exchange information effectively. A prime example is the HTTP protocol, which specifies the precise manner in which a web browser should formulate requests for information from a web server.

It is important to note that the upper layers remain agnostic regarding the implementation details of the lower layers, provided that the lower layers deliver the expected services. For example, HTTP does not concern itself with whether the underlying connection utilizes wireless technology, an RJ-45 Ethernet cable, a smartphone interface, or some combination of multiple transmission media. Similarly, the lower layers, at least in theory, do not concern themselves with the specific content or payload carried by the application layer.

For the purposes of our current discussion, our primary focus will be on the internet layer and the transport layer.

!!! note title="How many layers are needed" The practice of organizing protocols into four layers is commonly referred to as the TCP/IP model (named after the two most fundamental protocols within the suite) or the Internet Protocol Suite. Frequently, the link layer is further subdivided into two separate layers—the physical layer and the data link layer—resulting in a five-layer model rather than a four-layer one.

Alternatively, we may reference the Open Systems Interconnection model (commonly known as the OSI model), which defines a more granular structure consisting of seven distinct layers.

Internet layer and IP protocol¶

The central protocol operating at the network layer is the Internet Protocol (IP). An IP packet is composed of two primary components: the payload, which carries the actual data, and the header, which contains control information. A helpful real-world analogy would be a traditional letter: the header corresponds to the envelope, while the payload represents the contents of the letter itself.

This protocol establishes several foundational elements essential for network communication.

First and foremost, every device participating in the network is assigned a unique IP address. In the IPv4 specification, an address consists of four bytes, with each byte representing a numerical value ranging from 0 to 255. Partly as a consequence of inefficient address management practices, the pool of available IPv4 addresses utilizing only four bytes has been nearly exhausted. To address this limitation, a newer protocol, IPv6, was introduced, employing 32 bytes for address representation. Although both protocols are currently in use, IPv4 remains the predominant protocol as of the year 2020.

IP Header

The IP protocol permits the fragmentation of data packets when the underlying transmission layer imposes a maximum packet size that is smaller than the payload. In other words, if the payload exceeds the capacity of what the link layer can transmit within a single packet, the IP protocol will divide the packet into smaller fragments and transmit each fragment independently. The maximum allowable size for a single IP packet is 65,535 bytes, of which 20 bytes are reserved for the IP header itself. Early, naive implementations of IP fragmentation mechanisms inadvertently enabled certain exploits, including denial-of-service attacks.

Additionally, IP incorporates a checksum mechanism designed to verify that the payload has not become corrupted during transmission. It is crucial to understand that the purpose of this checksum is to safeguard against non-malicious errors that may arise from factors such as a noisy or unreliable underlying communication channel. If an attacker deliberately modifies a packet, however, it becomes trivial for them to recalculate and insert a new, valid checksum corresponding to the altered packet.

When a device—whether it is an end host or an intermediate router—detects a corrupted IP packet, the packet is discarded, and an error message is transmitted back to the source.

It is worth noting that the IP protocol does not directly specify the method by which a packet should be delivered. Instead, this decision is made by an IP routing algorithm executing on the router. The fundamental principle underlying such an algorithm is that a router examines the destination IP address and compares it against entries in its routing table. This routing table can be conceptualized as a collection of devices (either routers or end hosts) that are connected to the router, with each device associated with a specific range of IP addresses. If the target IP address falls within one of these ranges, the packet is forwarded to the next router (or host) associated with that particular range. Routing tables may be configured statically, meaning they are manually written by administrators, or they may be dynamic, utilizing various protocols to automatically update and maintain their contents.

Internet routing is deliberately designed so that if the most common or optimal communication pathway becomes disrupted, a router will attempt to establish an alternative, albeit potentially less efficient, route. However, if all available communication lines are simultaneously disrupted, there is no mechanism within the protocol to restore the connection.

The IP header contains a field labeled "protocol," which indicates the type of payload being carried. For instance, this field may specify that the payload consists of a TCP packet (a protocol we will examine in greater detail shortly), or it may indicate packets associated with route discovery algorithms.

A particularly notable example among these auxiliary protocols is the Internet Control Message Protocol (ICMP). This protocol is utilized for transmitting error messages and conveying network-related communication information. The most widely recognized—and occasionally infamous—application of ICMP is the Ping utility, a network diagnostic tool used to determine whether a specific host is reachable and to measure the time required to reach it. A classic denial-of-service (DoS) attack known as Ping Flood involves overwhelming a victim's system with a high volume of ping requests in an attempt to exhaust its resources. Another attack, referred to as Ping of Death, is actually an IP fragmentation attack that utilizes ICMP as the payload.

The IP protocol remains largely indifferent to the specifics of the lower layers, except when determining whether packet fragmentation is necessary. In fact, IP could theoretically be implemented over virtually any transmission medium—even carrier pigeons.

Transport layer: TCP and UDP¶

The IP protocol lacks several essential features that render it unsuitable for direct use in many application scenarios.

Firstly, IP does not provide a mechanism to distinguish between different applications running on the same host. For example, if a user maintains an open SSH shell connection while simultaneously downloading a web page through a browser, IP offers no means to direct incoming data to the web browser rather than to the shell application.

Secondly, IP is inherently stateless, meaning that each packet is processed independently of all others. Consequently, while IP does offer some protection against corrupted packets via its checksum mechanism, it provides no guarantee that packets will arrive in the order in which they were sent. Furthermore, packets may be duplicated during transmission. Finally, if a packet is lost for any reason, IP does not incorporate a reliable mechanism to automatically retransmit the missing packet.

The Transmission Control Protocol (TCP) was developed specifically to address these shortcomings.

A TCP connection is established between two communicating agents: a server, which waits passively for incoming connection requests, and a client, which actively initiates the connection. In a typical scenario, the server might be a web server, while the client is a web browser.

TCP maintains state information: upon initiating a connection, the client and server engage in a handshake protocol during which several packets are exchanged back and forth to ensure that both parties are synchronized and agree on the parameters of the connection. The integrity of the data stream is preserved through the use of sequence counters maintained on both ends, along with the transmission of an acknowledgment packet each time a data packet is successfully received. If the sender does not receive an acknowledgment for a transmitted data packet within a predefined time window, it will automatically retransmit that packet.

TCP also introduces the concept of a port: each connection involves two ports, one assigned to each communicating party. A port is represented as an integer value ranging from 0 to 65,535. A server typically listens for new connection requests on a well-known, standardized port. Default ports for widely adopted protocols are well-established and are usually represented by small numerical values; for example, HTTP conventionally uses port 80. A client also requires its own port for the connection. These client-side ports are typically large numerical values that are automatically selected by the operating system based on current port availability.

TCP Header¶

A diagram illustrating the structure of the TCP header, with the port number fields specifically highlighted.

TCP provides a reliable, ordered, and error-checked data stream connection between two devices. The primary drawback of this protocol is the significant overhead it introduces, particularly due to the necessity of transmitting acknowledgment packets. This added latency may be unacceptable for certain real-time applications, such as online gaming or live video streaming. An alternative protocol to TCP is the User Datagram Protocol (UDP), a much simpler protocol that provides port numbers and an optional checksum as additional services, but offers none of the reliability, ordering, or flow control features of TCP.

UDP Header¶

A diagram illustrating the structure of the UDP header, with the port number fields specifically highlighted.

!!! note title="LAN and NAT" The majority of Internet users, whether connecting from home or workplace environments, are not directly attached to the global Internet. Instead, they connect to a local area network (LAN). This local network is then connected to the broader Internet via a router. In a typical home setting, this router often doubles as a WiFi access point. Such a device frequently provides several additional services, including:

- A firewall, which selectively blocks incoming (and sometimes outgoing) connections based on predefined rules.
- A DHCP server, which dynamically assigns new IP addresses to devices as they join the LAN.
- A NAT (Network Address Translation) service, which maps combinations of local IP addresses and TCP/UDP port numbers to a single global IP address and corresponding ports. This functionality is particularly useful when an Internet service provider (ISP) allocates only one public IP address to a customer, yet the customer wishes to connect multiple devices simultaneously. NAT enables users to maintain locally unique IP addresses that are all mapped to the same globally visible IP address. This mapping is accomplished by utilizing distinct TCP or UDP port numbers to differentiate between the various internal connections.

Internet and security¶

Neither the TCP nor the IP protocol incorporates any built-in security mechanisms. This implies that, unless additional protective measures are implemented, traffic data can be intercepted or, even more concerningly, modified during transmission. This type of threat is commonly referred to as a man-in-the-middle attack. Modifying traffic in this manner can be executed with extreme sophistication; for example, an attacker could inject malicious JavaScript code into a web browser when a user visits third-party websites.

Data encryption can be implemented at the link layer. For instance, WiFi networks can utilize the WPA2 standard to encrypt their transmitted traffic. Similarly, data transmitted over cellular connections is typically encrypted. However, Ethernet—a common protocol for cable-based local area networks—does not inherently provide any protection. Moreover, encryption applied at the link layer does not guarantee that the data will remain encrypted throughout its entire journey to the destination host, particularly if the underlying transmission medium changes at any point during the communication.

For these reasons, encryption is more commonly implemented at the application layer, where secure protocols replace their plain-text counterparts. Representative examples include:

• SSH used in place of Telnet for establishing secure remote shell connections
• HTTPS used in place of HTTP for communicating securely with web servers
• SFTP used in place of FTP for transmitting files with encryption

These protocols establish a secure, encrypted channel between two devices, ensuring that sensitive data cannot be observed by unauthorized parties. However, certain metadata remains visible: an observer can still see the source and destination addresses as well as the port numbers. In other words, a man-in-the-middle attacker can determine that a user has contacted a particular server and can likely infer the requested service based on the port number, but the actual payload content remains concealed.

It is also possible to utilize the encryption protocol known as TLS (the same protocol employed by HTTPS) to encode standard IP traffic. In this scenario, the user connects to a specialized server and establishes a secure, encrypted channel. Through this channel, the user transmits IP data packets. The server then decrypts the stream and forwards the original packets to the destination address specified within them. Additionally, the server modifies the TCP/IP headers (in a manner similar to NAT) so that the connection appears to originate from the server itself. This arrangement also enables the user to receive replies, as the server will forward any incoming responses back to the user through the secure channel.

This approach essentially constitutes a form of Virtual Private Network (VPN). It is important to note that in this configuration, we are encapsulating an IP packet within a TLS stream, which itself operates over TCP, which in turn relies on IP. The presence of the TCP protocol in this stack can potentially lead to performance issues due to compounded acknowledgment mechanisms and latency. Because of these concerns, IP Security (IPSec) is more commonly employed for VPN implementations. IPSec data transmission protocols are implemented directly over IP (analogous to how TCP or ICMP operate) and are considered to reside at the network layer. Since the TCP protocol is not interposed between the native IP packets and the encrypted payload, the associated latency issues are significantly reduced.

It is crucial to understand that the payload is decrypted at the endpoint of the VPN tunnel before it is forwarded to its ultimate destination. Because the IP address is also modified during this process, an external observer cannot deduce the user's identity based solely on IP information. However, any sensitive information that is not separately encrypted within the payload itself (such as passwords transmitted over plain HTTP) can potentially be read after the traffic exits the VPN tunnel.

Encrypt, decrypt¶

Encryption algorithms¶

Encryption methods can be broadly classified into three distinct categories:

• methods that do not utilize a key
• methods relying on a symmetric concealed key
• methods utilizing an asymmetric public/private key pair

The "security" provided by an encryption technique that lacks a key relies entirely on the concept that the decryption algorithm itself remains unknown. Possibly the most widely recognized example of such a method is ROT13, in which letters are shifted by 13 positions and wrap around the alphabet. As a result, for instance, the letter 'a' is transformed into 'n'. Algorithms that do not use keys are not regarded as secure, and they should not be employed for any purpose other than newspaper puzzles.

Algorithms that rely on a symmetric hidden key operate on the assumption that both the encryption and decryption algorithms are publicly known, but both communicating parties share a secret key, ensuring that without this key, decryption is extremely challenging. Perhaps the most famous example of such a method is the Caesar cipher, named after Julius Caesar. This cipher involves shifting letters by a specific quantity and wrapping around the alphabet. In this case, the key is an integer that specifies the amount of the shift. ROT13 is actually a specific instance of the Caesar cipher where the shift is permanently fixed at 13. Julius Caesar himself utilized a shift of 3, whereas his nephew Augustus employed a shift of 1.

It is critically important that the number of possible candidates for a key is extremely large, rendering a brute-force attack impossible. For instance, the Caesar cipher can be broken easily, even without the aid of a computer, because there are only 26 possible options. A more modern example is the Content Scrambling System (CSS) for DVDs, which was introduced in 1996 and compromised in 1999. CSS utilized a 40-bit key, leaving 2 to the power of 40 possible options for a key value. The reason for such a "small" key was that, at the time, the United States had restrictions on exporting strong cryptographic methods (in modern times, these restrictions have been relaxed, though not entirely eliminated). The short key length permitted a brute-force attack, successfully breaking the encryption in 17 hours using a computer available at that time.

The primary issue with utilizing a symmetric key is that both parties must possess the key beforehand. To resolve this issue, algorithms based on a public/private key pair are employed. The fundamental concept is as follows: A recipient possesses two keys, one public and one private. A sender encrypts a message using an algorithm along with the public key. This message can subsequently only be decrypted using the private key. Consequently, only the intended recipient is capable of decrypting the message.

Since the public key is known, it is vital for the encryption algorithm that one cannot easily deduce the private key from the public key. In other words, there is no significantly more efficient method of deriving a private key than simply attempting a brute-force attack. If both the public and private keys are sufficiently long, then a brute-force attack becomes unfeasible. The most widely used and recognized public/private encryption method is RSA, where the security is derived from the common assumption that the factorization of very large integers is computationally very expensive.

Asymmetric encryption is often not efficient enough for encrypting large streams of data. Therefore, in practice, it is used solely to establish a symmetric hidden key that is valid only for a single session, and this key is then utilized with a more efficient algorithm based on a symmetric hidden key.

Cryptographic hashes¶

Modern computer systems rarely store passwords in clear text. Otherwise, if the file containing the passwords is compromised, the attacker acquires a list of plain-text passwords for that specific system—a significant security breach. Instead, passwords are stored as hashes. They are scrambled using a known hash function. The key distinction here is that a hash cannot be decrypted. When a user attempts to log in to the system, the stored password is not decrypted. Instead, the user input is hashed, and the resulting hashes are compared.

Since hashes are deterministic, the same password will always produce the same hash. This leads to the possibility that an attacker can construct a large library of reversed hashes by simply precomputing them. To combat this attack, passwords are typically salted: a random string, unique to each user, is appended to a password before it is hashed. The salt is stored in plain text, so that it can be utilized when password verification is required.

Currently, the most prominent hash function family is SHA-2. A notable hash function, MD5, has also been used for hashing sensitive data. However, it has been severely compromised and should not be used for any security application. MD5 is still useful, for example, in providing a unique identifier for documents.

Certificates¶

While encryption ensures that a third party cannot eavesdrop on the communication, it is still vulnerable to a man-in-the-middle attack: the third party can pretend to be the server that the user wishes to contact. The attack is executed as follows. Consider two parties, Alice and Bob, and an attacker named Melissa. Alice wishes to send a message safely to Bob, and Melissa wishes to intercept the message.

1. Melissa makes Alice believe that she is Bob.
2. Alice asks Bob (Melissa in disguise) for a public key. Instead, Melissa provides Alice with her own public key.
3. Alice encrypts the information with the spoofed key and sends it back to Melissa.
4. Melissa decrypts the information using her secret key. The crux of the problem is that Alice cannot verify whether the public key actually belongs to Bob. To solve this problem, certificates are used.

Certificates function as follows. Consider that there is an additional party, Benedict. Alice wants to send an encrypted message but is not sure whether Bob's public key is truly his. However, Alice knows and trusts Benedict's public key. The verification consists of the following steps:

1. Bob asks Benedict to sign his public key.
2. Benedict uses Bob's public key and his identity to construct a signature, that is encrypted with Benedict's secret key, and sends Bob the signature.
3. Upon request, Bob sends Alice the certificate, that is, his information and the encrypted signature.
4. Alice decrypts the signature using Benedict's public key and verifies that the information in the certificate matches the decrypted signature.
5. Alice checks that the identity in the certificate is indeed Bob.
6. Alice can now trust the public key provided in Bob's message.

Let's examine the sequence of these events more closely. First, note that the signature is created by encrypting using a secret key, instead of a public key. Since Benedict is the only one that knows the secret key, he is the only one that can produce the signature. The signature contains Bob's identity as well as the public key, so Alice can trust that the information is correct by trusting Benedict's action.

This is how certificates work. In the above example, Benedict is known as a Certificate Authority (CA), Bob is a web server, and Alice is a normal user. Certificate Authorities are companies that provide certificates for web servers, essentially giving them the means to prove who they are.

Melissa cannot fake being Bob, because she needs to provide a fake certificate for Bob. She has 3 options, none of which will succeed:

She can use Bob's certificate, but she cannot decrypt the incoming traffic because Alice will encrypt the data with Bob's public key. She can obtain her own valid certificate, but Alice will notice that the certificate is Melissa's and not Bob's. She can modify the certificate by replacing Bob's key with her own but the content no longer matches the signature, so Alice rejects the tampered signature. The key assumption here is that Alice trusts Benedict. In practice, a computer has a list of trusted certificate authorities, that are obtained via a secure channel, for example, when installing a new operating system.

In practice, there may be intermediate parties, meaning Bob doesn't use Benedict directly, instead he asks a mediator, say William, to issue a certificate. William has his own certificate that he has obtained from Benedict. Alice then needs to verify both Bob's and William's certificates.

A similar scheme is also used for signing executables.