How to Think Like a Computer Scientist: Learning with Python 2nd Edition/Strings

= Strings =

7.1 A compound data type
So far we have seen five types: int, float, bool, NoneType and str. Strings are qualitatively different from the other four because they are made up of smaller pieces --- characters.

Types that comprise smaller pieces are called compound data types. Depending on what we are doing, we may want to treat a compound data type as a single thing, or we may want to access its parts. This ambiguity is useful.

The bracket operator selects a single character from a string:

The expression fruit[1] selects character number 1 from fruit. The variable letter refers to the result. When we display <tt>letter</tt>, we get a surprise:

The first letter of <tt>&quot;banana&quot;</tt> is not <tt>a</tt>, unless you are a computer scientist. For perverse reasons, computer scientists always start counting from zero. The 0th letter ( zero-eth ) of <tt>&quot;banana&quot;</tt> is <tt>b</tt>. The 1th letter ( one-eth ) is <tt>a</tt>, and the 2th ( two-eth ) letter is <tt>n</tt>.

If you want the zero-eth letter of a string, you just put 0, or any expression with the value 0, in the brackets:

The expression in brackets is called an index. An index specifies a member of an ordered set, in this case the set of characters in the string. The index indicates which one you want, hence the name. It can be any integer expression.

7.2 Length
The <tt>len</tt> function returns the number of characters in a string:

To get the last letter of a string, you might be tempted to try something like this:

That won't work. It causes the runtime error <tt>IndexError: string index out of range</tt>. The reason is that there is no 6th letter in <tt>&quot;banana&quot;</tt>. Since we started counting at zero, the six letters are numbered 0 to 5. To get the last character, we have to subtract 1 from <tt>length</tt>:

Alternatively, we can use negative indices, which count backward from the end of the string. The expression <tt>fruit[-1]</tt> yields the last letter, <tt>fruit[-2]</tt> yields the second to last, and so on.

7.3 Traversal and the <tt>for</tt> loop
A lot of computations involve processing a string one character at a time. Often they start at the beginning, select each character in turn, do something to it, and continue until the end. This pattern of processing is called a traversal. One way to encode a traversal is with a <tt>while</tt> statement:

This loop traverses the string and displays each letter on a line by itself. The loop condition is <tt>index &lt; len(fruit)</tt>, so when <tt>index</tt> is equal to the length of the string, the condition is false, and the body of the loop is not executed. The last character accessed is the one with the index <tt>len(fruit)-1</tt>, which is the last character in the string.

Using an index to traverse a set of values is so common that Python provides an alternative, simpler syntax --- the <tt>for</tt> loop:

Each time through the loop, the next character in the string is assigned to the variable <tt>letter</tt>. The loop continues until no characters are left.

The following example shows how to use concatenation and a <tt>for</tt> loop to generate an abecedarian series. Abecedarian refers to a series or list in which the elements appear in alphabetical order. For example, in Robert McCloskey's book Make Way for Ducklings, the names of the ducklings are Jack, Kack, Lack, Mack, Nack, Ouack, Pack, and Quack. This loop outputs these names in order:

The output of this program is:

Jack Kack Lack Mack Nack Oack Pack Qack Of course, that's not quite right because Ouack and Quack are misspelled. You'll fix this as an exercise below.

7.4 String slices
A substring of a string is called a slice. Selecting a slice is similar to selecting a character:

The operator <tt>[n:m]</tt> returns the part of the string from the n-eth character to the m-eth character, including the first but excluding the last. This behavior is counterintuitive; it makes more sense if you imagine the indices pointing between the characters, as in the following diagram:



If you omit the first index (before the colon), the slice starts at the beginning of the string. If you omit the second index, the slice goes to the end of the string. Thus:

What do you think <tt>s[:]</tt> means?

String comparison
The comparison operators work on strings. To see if two strings are equal:

Other comparison operations are useful for putting words in lexigraphical order_:

This is similar to the alphabetical order you would use with a dictionary, except that all the uppercase letters come before all the lowercase letters. As a result:

A common way to address this problem is to convert strings to a standard format, such as all lowercase, before performing the comparison. A more difficult problem is making the program realize that zebras are not fruit.

Strings are immutable
It is tempting to use the <tt>[]</tt> operator on the left side of an assignment, with the intention of changing a character in a string. For example:

Instead of producing the output <tt>Jello, world!</tt>, this code produces the runtime error <tt>TypeError: 'str' object doesn't support item assignment</tt>.

Strings are immutable, which means you can't change an existing string. The best you can do is create a new string that is a variation on the original:

The solution here is to concatenate a new first letter onto a slice of <tt>greeting</tt>. This operation has no effect on the original string.

The <tt>in</tt> operator
The <tt>in</tt> operator tests if one string is a substring of another:

Note that a string is a substring of itself:

Combining the <tt>in</tt> operator with string concatenation using <tt>+</tt>, we can write a function that removes all the vowels from a string:

Test this function to confirm that it does what we wanted it to do.

7.8 A <tt>find</tt> function
What does the following function do?

In a sense, <tt>find</tt> is the opposite of the <tt>[]</tt> operator. Instead of taking an index and extracting the corresponding character, it takes a character and finds the index where that character appears. If the character is not found, the function returns <tt>-1</tt>.

This is the first example we have seen of a <tt>return</tt> statement inside a loop. If <tt>strng[index] == ch</tt>, the function returns immediately, breaking out of the loop prematurely.

If the character doesn't appear in the string, then the program exits the loop normally and returns <tt>-1</tt>.

This pattern of computation is sometimes called a eureka traversal because as soon as we find what we are looking for, we can cry Eureka! and stop looking.

Looping and counting
The following program counts the number of times the letter <tt>a</tt> appears in a string, and is another example of the counter pattern introduced in :ref:`counting`:

7.10 Optional parameters
To find the locations of the second or third occurrence of a character in a string, we can modify the <tt>find</tt> function, adding a third parameter for the starting position in the search string:

The call <tt>find2('banana', 'a', 2)</tt> now returns <tt>3</tt>, the index of the first occurrence of 'a' in 'banana' after index 2. What does <tt>find2('banana', 'n', 3)</tt> return? If you said, 4, there is a good chance you understand how <tt>find2</tt> works.

Better still, we can combine <tt>find</tt> and <tt>find2</tt> using an optional parameter:

The call <tt>find('banana', 'a', 2)</tt> to this version of <tt>find</tt> behaves just like <tt>find2</tt>, while in the call <tt>find('banana', 'a')</tt>, <tt>start</tt> will be set to the default value of <tt>0</tt>.

Adding another optional parameter to <tt>find</tt> makes it search both forward and backward:

Passing in a value of <tt>len(strng)-1</tt> for start and <tt>-1</tt> for <tt>step</tt> will make it search toward the beginning of the string instead of the end. Note that we needed to check for a lower bound for <tt>index</tt> in the while loop as well as an upper bound to accommodate this change.

The <tt>string</tt> module
The <tt>string</tt> module contains useful functions that manipulate strings. As usual, we have to import the module before we can use it:

To see what is inside it, use the <tt>dir</tt> function with the module name as an argument.

which will return the list of items inside the string module:

<tt>['Template', '_TemplateMetaclass', '__builtins__', '__doc__', '__file__', '__name__', '_float', '_idmap', '_idmapL', '_int', '_long', '_multimap', '_re', 'ascii_letters', 'ascii_lowercase', 'ascii_uppercase', 'atof', 'atof_error', 'atoi', 'atoi_error', 'atol', 'atol_error', 'capitalize', 'capwords', 'center', 'count', 'digits', 'expandtabs', 'find', 'hexdigits', 'index', 'index_error', 'join', 'joinfields', 'letters', 'ljust', 'lower', 'lowercase', 'lstrip', 'maketrans', 'octdigits', 'printable', 'punctuation', 'replace', 'rfind', 'rindex', 'rjust', 'rsplit', 'rstrip', 'split', 'splitfields', 'strip', 'swapcase', 'translate', 'upper', 'uppercase', 'whitespace', 'zfill']</tt>

To find out more about an item in this list, we can use the <tt>type</tt> command. We need to specify the module name followed by the item using dot notation.

Since <tt>string.digits</tt> is a string, we can print it to see what it contains:

Not surprisingly, it contains each of the decimal digits.

<tt>string.find</tt> is a function which does much the same thing as the function we wrote. To find out more about it, we can print out its docstring, <tt>__doc__</tt>, which contains documentation on the function:

The parameters in square brackets are optional parameters. We can use <tt>string.find</tt> much as we did our own <tt>find</tt>:

This example demonstrates one of the benefits of modules --- they help avoid collisions between the names of built-in functions and user-defined functions. By using dot notation we can specify which version of <tt>find</tt> we want.

Actually, <tt>string.find</tt> is more general than our version. it can find substrings, not just characters:

Like ours, it takes an additional argument that specifies the index at which it should start:

Unlike ours, its second optional parameter specifies the index at which the search should end:

In this example, the search fails because the letter b does not appear in the index range from <tt>1</tt> to <tt>2</tt> (not including <tt>2</tt>).

Character classification
It is often helpful to examine a character and test whether it is upper- or lowercase, or whether it is a character or a digit. The <tt>string</tt> module provides several constants that are useful for these purposes. One of these, <tt>string.digits</tt>, we have already seen.

The string <tt>string.lowercase</tt> contains all of the letters that the system considers to be lowercase. Similarly, <tt>string.uppercase</tt> contains all of the uppercase letters. Try the following and see what you get:

We can use these constants and <tt>find</tt> to classify characters. For example, if <tt>find(lowercase, ch)</tt> returns a value other than <tt>-1</tt>, then <tt>ch</tt> must be lowercase:

Alternatively, we can take advantage of the <tt>in</tt> operator:

As yet another alternative, we can use the comparison operator:

If <tt>ch</tt> is between a and z, it must be a lowercase letter.

Another constant defined in the <tt>string</tt> module may surprise you when you print it:

Whitespace characters move the cursor without printing anything. They create the white space between visible characters (at least on white paper). The constant <tt>string.whitespace</tt> contains all the whitespace characters, including space, tab (<tt>\t</tt>), and newline (<tt>\n</tt>).

There are other useful functions in the <tt>string</tt> module, but this book isn't intended to be a reference manual. On the other hand, the Python Library Reference is. Along with a wealth of other documentation, it's available from the Python website, [http://www.python.org http://www.python.org]_.

String formatting
The most concise and powerful way to format a string in Python is to use the string formatting operator, <tt>%</tt>, together with Python's string formatting operations. To see how this works, let's start with a few examples:

The syntax for the string formatting operation looks like this:

It begins with a format which contains a sequence of characters and conversion specifications. Conversion specifications start with a <tt>%</tt> operator. Following the format string is a single <tt>%</tt> and then a sequence of values, one per conversion specification, separated by commas and enclosed in parenthesis. The parenthesis are optional if there is only a single value.

In the first example above, there is a single conversion specification, <tt>%s</tt>, which indicates a string. The single value, <tt>&quot;Arthur&quot;</tt>, maps to it, and is not enclosed in parenthesis.

In the second example, <tt>name</tt> has string value, <tt>&quot;Alice&quot;</tt>, and <tt>age</tt> has integer value, <tt>10</tt>. These map to the two conversion specifications, <tt>%s</tt> and <tt>%d</tt>. The <tt>d</tt> in the second conversion specification indicates that the value is a decimal integer.

In the third example variables <tt>n1</tt> and <tt>n2</tt> have integer values <tt>4</tt> and <tt>5</tt> respectively. There are four conversion specifications in the format string: three <tt>%d</tt>'s and a <tt>%f</tt>. The <tt>f</tt> indicates that the value should be represented as a floating point number. The four values that map to the four conversion specifications are: <tt>2**10</tt>, <tt>n1</tt>, <tt>n2</tt>, and <tt>n1 * n2</tt>.

<tt>s</tt>, <tt>d</tt>, and <tt>f</tt> are all the conversion types we will need for this book. To see a complete list, see the String Formatting Operations_ section of the Python Library Reference.

The following example illustrates the real utility of string formatting:

This program prints out a table of various powers of the numbers from 1 to 10. In its current form it relies on the tab character ( <tt>\t</tt>) to align the columns of values, but this breaks down when the values in the table get larger than the 8 character tab width:

i      i**2    i**3    i**5    i**10   i**20 1      1       1       1       1       1 2       4       8       32      1024    1048576 3       9       27      243     59049   3486784401 4       16      64      1024    1048576         1099511627776 5       25      125     3125    9765625         95367431640625 6       36      216     7776    60466176        3656158440062976 7       49      343     16807   282475249       79792266297612001 8       64      512     32768   1073741824      1152921504606846976 9       81      729     59049   3486784401      12157665459056928801 10      100     1000    100000  10000000000     100000000000000000000 One possible solution would be to change the tab width, but the first column already has more space than it needs. The best solution would be to set the width of each column independently. As you may have guessed by now, string formatting provides the solution:

Running this version produces the following output:

i  i**2 i**3  i**5    i**10        i**20 1  1    1     1       1            1              2   4    8     32      1024         1048576        3   9    27    243     59049        3486784401     4   16   64    1024    1048576      1099511627776  5   25   125   3125    9765625      95367431640625 6   36   216   7776    60466176     3656158440062976 7   49   343   16807   282475249    79792266297612001 8   64   512   32768   1073741824   1152921504606846976 9   81   729   59049   3486784401   12157665459056928801 10  100  1000  100000  10000000000  100000000000000000000 The <tt>-</tt> after each <tt>%</tt> in the converstion specifications indicates left justification. The numerical values specify the minimum length, so <tt>%-13d</tt> is a left justified number at least 13 characters wide.

Summary and First Exercises
This chapter introduced a lot of new ideas. The following summary and set of exercises may prove helpful in remembering what you learned:

Exercises:

<ol> <li> Write the Python interpreter's evaluation to each of the following expressions: <ul> <li>&gt;&gt;&gt; 'Python'[1]</li> <li>&gt;&gt;&gt; &quot;Strings are sequences of characters.&quot;[5]</li> <li>&gt;&gt;&gt; len(&quot;wonderful&quot;)</li> <li>&gt;&gt;&gt; 'Mystery'[:4]</li> <li>&gt;&gt;&gt; 'p' in 'Pinapple'</li> <li>&gt;&gt;&gt; 'apple' in 'Pinapple'</li> <li>&gt;&gt;&gt; 'pear' in 'Pinapple'</li> <li>&gt;&gt;&gt; 'apple' &gt; 'pinapple'</li> <li>&gt;&gt;&gt; 'pinapple' &lt; 'Peach'</li></ul> </li> <li><dl> <dt>Write Python code to make each of the following doctests pass:</dt> <dd><dl> <dt>*</dt> <dd></dd> <dt>*</dt> <dd></dd> <dt>*</dt> <dd></dd></dl> </dd></dl> </li></ol>

Question 1
Modify: so that Ouack and Quack are spelled correctly.

Question 2
Encapsulate:

in a function named, and generalize it so that it accepts the string and the letter as arguments.

Question 3
Now rewrite the count_letters function so that instead of traversing the string, it repeatedly calls find (the version from Optional parameters), with the optional third parameter to locate new occurences of the letter being counted.

Question 4
Which version of <tt>is_lower</tt> do you think will be fastest? Can you think of other reasons besides speed to prefer one version or the other?

Question 5
<ul><li> Create a file named <tt>stringtools.py</tt> and put the following in it:

Add a function body to <tt>reverse</tt> to make the doctests pass.</li> <li> Add <tt>mirror</tt> to <tt>stringtools.py</tt>.

Write a function body for it that will make it work as indicated by the doctests.</li> <li> Include <tt>remove_letter</tt> in <tt>stringtools.py</tt>.

Write a function body for it that will make it work as indicated by the doctests.</li> <li> Finally, add bodies to each of the following functions, one at a time

until all the doctests pass.</li> <li> Try each of the following formatted string operations in a Python shell and record the results: <ol> <li>&quot;%s %d %f&quot; % (5, 5, 5)</li> <li>&quot;%-.2f&quot; % 3</li> <li>&quot;%-10.2f%-10.2f&quot; % (7, 1.0/2)</li> <li>print &quot; $%5.2fn $%5.2fn $%5.2f&quot; % (3, 4.5, 11.2)</li> </ol> </li> <li> The following formatted strings have errors. Fix them: <ol> <li>&quot;%s %s %s %s&quot; % ('this', 'that', 'something')</li> <li>&quot;%s %s %s&quot; % ('yes', 'no', 'up', 'down')</li> <li>&quot;%d %f %f&quot; % (3, 3, 'three')</li> </ol> </ul>