Managing Resources in Python 관련

Context Managers and Python's with Statement

---

In this step-by-step tutorial, you'll learn what the Python with statement is and how to use it with existing context managers. You'll also learn how to create your own context managers.

• The try … finally Approach
• The with Statement Approach

---

Context Managers and Python's with Statement

In this step-by-step tutorial, you'll learn what the Python with statement is and how to use it with existing context managers. You'll also learn how to create your own context managers.

One common problem you’ll face in programming is how to properly manage external resources, such as files, locks, and network connections. Sometimes, a program will retain those resources forever, even if you no longer need them. This kind of issue is called a memory leak because the available memory gets reduced every time you create and open a new instance of a given resource without closing an existing one.

Managing resources properly is often a tricky problem. It requires both a setup phase and a teardown phase. The latter phase requires you to perform some cleanup actions, such as closing a file, releasing a lock, or closing a network connection. If you forget to perform these cleanup actions, then your application keeps the resource alive. This might compromise valuable system resources, such as memory and network bandwidth.

For example, a common problem that can arise when developers are working with databases is when a program keeps creating new connections without releasing or reusing them. In that case, the database back end can stop accepting new connections. This might require an admin to log in and manually kill those stale connections to make the database usable again.

Another frequent issue shows up when developers are working with files. Writing text to files is usually a buffered operation. This means that calling .write() on a file won’t immediately result in writing text to the physical file but to a temporary buffer. Sometimes, when the buffer isn’t full and developers forget to call .close(), part of the data can be lost forever.

Another possibility is that your application runs into errors or exceptions that cause the control flow to bypass the code responsible for releasing the resource at hand. Here’s an example in which you use open() to write some text to a file:

file = open("hello.txt", "w")
file.write("Hello, World!")
file.close()

This implementation doesn’t guarantee the file will be closed if an exception occurs during the .write() call. In this case, the code will never call .close(), and therefore your program might leak a file descriptor.

In Python, you can use two general approaches to deal with resource management. You can wrap your code in:

1. A try … finally construct
2. A with construct

The first approach is quite general and allows you to provide setup and teardown code to manage any kind of resource. However, it’s a little bit verbose. Also, what if you forget any cleanup actions?

The second approach provides a straightforward way to provide and reuse setup and teardown code. In this case, you’ll have the limitation that the with statement only works with context managers. In the next two sections, you’ll learn how to use both approaches in your code.

---

The try … finally Approach

Working with files is probably the most common example of resource management in programming. In Python, you can use a try … finally statement to handle opening and closing files properly:

# Safely open the file
file = open("hello.txt", "w")

try:
    file.write("Hello, World!")
finally:
    # Make sure to close the file after using it
    file.close()

In this example, you need to safely open the file hello.txt, which you can do by wrapping the call to open() in a try … except statement. Later, when you try to write to file, the finally clause will guarantee that file is properly closed, even if an exception occurs during the call to .write() in the try clause. You can use this pattern to handle setup and teardown logic when you’re managing external resources in Python.

The try block in the above example can potentially raise exceptions, such as AttributeError or NameError. You can handle those exceptions in an except clause like this:

# Safely open the file
file = open("hello.txt", "w")

try:
    file.write("Hello, World!")
except Exception as e:
    print(f"An error occurred while writing to the file: {e}")
finally:
    # Make sure to close the file after using it
    file.close()

In this example, you catch any potential exceptions that can occur while writing to the file. In real-life situations, you should use a specific exception type instead of the general Exception to prevent unknown errors from passing silently.

---

The with Statement Approach

The Python with statement creates a runtime context that allows you to run a group of statements under the control of a context manager. PEP 343 added the with statement to make it possible to factor out standard use cases of the try … finally statement.

Compared to traditional try … finally constructs, the with statement can make your code clearer, safer, and reusable. Many classes in the standard library support the with statement. A classic example of this is open(), which allows you to work with file objects using with.

To write a with statement, you need to use the following general syntax:

with expression as target_var:
    do_something(target_var)

The context manager object results from evaluating the expression after with. In other words, expression must return an object that implements the context management protocol. This protocol consists of two special methods:

1. .__enter__() is called by the with statement to enter the runtime context.
2. .__exit__() is called when the execution leaves the with code block.

The as specifier is optional. If you provide a target_var with as, then the return value of calling .__enter__() on the context manager object is bound to that variable.

Note

Some context managers return None from .__enter__() because they have no useful object to give back to the caller. In these cases, specifying a target_var makes no sense.

Here’s how the with statement proceeds when Python runs into it:

1. Call expression to obtain a context manager.
2. Store the context manager’s .__enter__() and .__exit__() methods for later use.
3. Call .__enter__() on the context manager and bind its return value to target_var if provided.
4. Execute the with code block.
5. Call .__exit__() on the context manager when the with code block finishes.

In this case, .__enter__(), typically provides the setup code. The with statement is a compound statement that starts a code block, like a conditional statement or a for loop. Inside this code block, you can run several statements. Typically, you use the with code block to manipulate target_var if applicable.

Once the with code block finishes, .__exit__() gets called. This method typically provides the teardown logic or cleanup code, such as calling .close() on an open file object. That’s why the with statement is so useful. It makes properly acquiring and releasing resources a breeze.

Here’s how to open your hello.txt file for writing using the with statement:

with open("hello.txt", mode="w") as file:
    file.write("Hello, World!")

When you run this with statement, open() returns an io.TextIOBase object. This object is also a context manager, so the with statement calls .__enter__() and assigns its return value to file. Then you can manipulate the file inside the with code block. When the block ends, .__exit__() automatically gets called and closes the file for you, even if an exception is raised inside the with block.

This with construct is shorter than its try … finally alternative, but it’s also less general, as you already saw. You can only use the with statement with objects that support the context management protocol, whereas try … finally allows you to perform cleanup actions for arbitrary objects without the need for supporting the context management protocol.

In Python 3.1 and later, the with statement supports multiple context managers. You can supply any number of context managers separated by commas:

with A() as a, B() as b:
    pass

This works like nested with statements but without nesting. This might be useful when you need to open two files at a time, the first for reading and the second for writing:

with open("input.txt") as in_file, open("output.txt", "w") as out_file:
    # Read content from input.txt
    # Transform the content
    # Write the transformed content to output.txt
    pass

In this example, you can add code for reading and transforming the content of input.txt. Then you write the final result to output.txt in the same code block.

Using multiple context managers in a single with has a drawback, though. If you use this feature, then you’ll probably break your line length limit. To work around this, you need to use backslashes (``) for line continuation, so you might end up with an ugly final result.

The with statement can make the code that deals with system resources more readable, reusable, and concise, not to mention safer. It helps avoid bugs and leaks by making it almost impossible to forget cleaning up, closing, and releasing a resource after you’re done with it.

Using with allows you to abstract away most of the resource handling logic. Instead of having to write an explicit try … finally statement with setup and teardown code each time, with takes care of that for you and avoids repetition.