CS253 Iterators

Array Traversal

You have an array, and you want to traverse (walk through) it.

int a[] = {2, 3, 5, 7, 11, 13, 17, 19};
for (int i=0; i!=8; ++i)
    cout << a[i] << ' ';

2 3 5 7 11 13 17 19

a[i] is the same as *(a+i).
- It requires addition and indirection.
I usually say i<8, rather than i!=8.
Similarly, I usually say i++ rather than ++i.
Patience.

Array Traversal via Pointers

Let’s do it with a pointer:

int a[] = {2, 3, 5, 7, 11, 13, 17, 19};
for (int *p = &a[0]; p != &a[8]; ++p)
    cout << *p << ' ';

2 3 5 7 11 13 17 19

That’s a bit faster.
Instead of a[i] (addition+indirection) we have just indirection.
Not really a radical improvement, however.

`vector` traversal

You can traverse a vector in the same way:

vector<int> a = {2, 3, 5, 7, 11, 13, 17, 19};
for (int *p = &a[0]; p != &a[8]; ++p)
    cout << *p << ' ';

2 3 5 7 11 13 17 19

or, avoiding the magic number 8:

vector<int> a = {2, 3, 5, 7, 11, 13, 17, 19};
for (int *p = &a[0]; p != &a[a.size()]; ++p)
    cout << *p << ' ';

2 3 5 7 11 13 17 19

Methods

vector<int> a = {2, 3, 5, 7, 11, 13, 17, 19};
for (vector<int>::iterator it = a.begin(); it != a.end(); ++it)
    cout << *it << ' ';

2 3 5 7 11 13 17 19

iterator is a type provided by the templated vector class.
What type is it?
- might be int *, might not
You don’t care what type it is. You know that:
- * works on it
- ++ works on it
- you can assign .begin() to it
- you can compare it to .end()

`auto` is your friend

This is prettier:

vector<int> a = {2, 3, 5, 7, 11, 13, 17, 19};
for (auto it = a.begin(); it != a.end(); ++it)
    cout << *it << ' ';

2 3 5 7 11 13 17 19

`for` loop

This is (nearly) exactly the same, and gorgeous:

vector<int> a = {2, 3, 5, 7, 11, 13, 17, 19};
for (auto v : a)
    cout << v << ' ';

2 3 5 7 11 13 17 19

The for loop is defined to use .begin() and .end(), just as the previous code.

Variations

Which is best? Why?

vector<int> a = {2, 3, 5, 7, 11, 13, 17, 19};
for (int v : a)
    cout << v << ' ';

2 3 5 7 11 13 17 19

vector<int> a = {2, 3, 5, 7, 11, 13, 17, 19};
for (const auto v : a)
    cout << v << ' ';

2 3 5 7 11 13 17 19

vector<int> a = {2, 3, 5, 7, 11, 13, 17, 19};
for (auto &v : a)
    cout << v << ' ';

2 3 5 7 11 13 17 19

vector<int> a = {2, 3, 5, 7, 11, 13, 17, 19};
for (const auto &v : a)
    cout << v << ' ';

2 3 5 7 11 13 17 19

Other containers

The same iterator code works for all STL containers.

forward_list<char> l = {'a', 'c', 'k', 'J'};
for (auto it = l.begin(); it != l.end(); ++it)
    cout << *it << ' ';

a c k J

unordered_set<string> u = {"a", "c", "k", "J"};
for (auto it = u.begin(); it != u.end(); ++it)
    cout << *it << ' ';

J k c a

set<char> s = {'a', 'c', 'k', 'J'};
for (auto it = s.begin(); it != s.end(); ++it)
    cout << *it << ' ';

J a c k

Of the top 100 ♀ + top 100 ♂ U.S. names, 1918–2017, the ones in ASCII order: Amy, Ann, Betty, Billy, Gary, Harry, Henry, Jack, Jerry, Kelly, Larry, Mary, Roy, Scott, and Terry.

`iterator` type

What type is set<int>::iterator?
- Who cares?
- It’s not an int *, that’s for sure!
- Probably a pointer to some non-user-visible data structure.
Same for list<int>::iterator and unordered_set<int>::iterator.
Most iterator types cannot be a native pointer.
- What happens when you increment a plain pointer?
- It goes to the next memory location.
- Is that what you want for a linked list, a tree, or a hash?
- It must be that operator++ is overloaded.
- Can’t do that for a plain pointer.
  - C++ is extensible, not mutable.
- Therefore, most iterators cannot be plain pointers.

Finding out

Sizes of various iterators:

cout << sizeof(int *)                    << '\n'
     << sizeof(list<int>::iterator)      << '\n'
     << sizeof(string::iterator)         << '\n'
     << sizeof(vector<int>::iterator)    << '\n'
     << sizeof(set<int>::iterator)       << '\n'
     << sizeof(array<int,100>::iterator) << '\n';

Finding out

A cheap way to find out what this implementation uses:

return string().end();

c.cc:1: error: cannot convert 'std::__cxx11::basic_string<char>::iterator' {aka 
   '__gnu_cxx::__normal_iterator<char*, std::__cxx11::basic_string<char> >'} to 
   'int' in return

return vector<int>().end();

c.cc:1: error: cannot convert 'std::vector<int>::iterator' {aka 
   '__gnu_cxx::__normal_iterator<int*, std::vector<int> >'} to 'int' in return

return set<int>().end();

c.cc:1: error: cannot convert 'std::set<int>::iterator' {aka 
   'std::_Rb_tree_const_iterator<int>'} to 'int' in return

return array<int,100>().end();

c.cc:1: error: invalid conversion from 'std::array<int, 100>::iterator' {aka 
   'int*'} to 'int'

Iterator classifications

BidirectionalIterator (list, set, unordered_set)
- ++, --, ==, !=, *, ->, =
RandomAccessIterator (std::array, vector, string)
- ++, --, +=int, -=int, +int, -int, iter-iter, ==, !=, <, <=, >, >=, *, ->, =
ForwardIterator (forward_list)
- ++, ==, !=, *, ->, =

`begin()` & `end()`

.begin() & .end() return iterators. Not necessarily pointers.
.begin() is, conceptually, a pointer to the first element of the container.
.end() is, conceptually, a pointer one past the end of the container.
It’s a half-open interval!

string s = "bonehead";
cout << "First char " << *s.begin() << '\n';
cout << "Badness: " << *s.end() << '\n';

First char b
Badness: ␀

string s = "genius";
cout << "First char " << *s.begin() << '\n';
cout << "Last char  " << *(s.end()-1) << '\n';

First char g
Last char  s

`.front()` & `.back()`

Some containers have .front() and .back(), which return references to the first and last elements.

list<double> c = {2.3, 5.7, 11.13, 17.19, 23.29};
cout << "First: " << c.front() << '\n';
cout << "Last:  " << c.back() << '\n';

First: 2.3
Last:  23.29

These are not iterators.

Comparisons, part one

This won’t work:

list<string> l = {"kappa", "alpha", "gamma"};
for (auto it = l.begin(); it < l.end(); ++it)
    cout << *it << ' ';

c.cc:2: error: no match for 'operator<' in 'it < 
   l.std::__cxx11::list<std::__cxx11::basic_string<char> >::end()' (operand 
   types are 'std::_List_iterator<std::__cxx11::basic_string<char> >' and 
   'std::__cxx11::list<std::__cxx11::basic_string<char> >::iterator' {aka 
   'std::_List_iterator<std::__cxx11::basic_string<char> >'})

Comparisons, part two

This will work:

list<string> l = {"kappa", "alpha", "gamma"};
for (auto it = l.begin(); it != l.end(); ++it)
    cout << *it << ' ';

kappa alpha gamma

list<>::iterator is a BidirectionalIterator, not a RandomAccessIterator, and so < isn’t defined. What would it compare? The addresses of the linked list nodes? That’s not useful.

Constructors

All containers accept a pair of iterators as ctor arguments. These do not have to be iterators for the same type of container.

char date[] = __DATE__;    // E.g., May 18 2024
string now(date, date+6);
cout << "month & day: " << now << '\n';

string day_of_month(now.begin()+4, now.begin()+6);
cout << "day: " << day_of_month << '\n';

multiset<int> ms(now.begin(), now.end());
for (auto n : ms)
    cout << n << ' ';
cout << '\n';

month & day: May 18
day: 18
32 49 56 77 97 121

`begin()` and `end()` functions

There are also free functions begin() and end(), which work on arrays (not pointers) and all standard containers:

char now[] = "08:31am Sat";
string current(now, now+10);
cout << current << '\n';

08:31am Sa

Oh dear, I counted wrong. Why am I counting!?

char now[] = "08:31am Sat";
string current(begin(now), end(now));
cout << current << '\n';

08:31am Sat␀

That didn’t work, either. What fresh hell is this? What is sizeof(now)?

Invalidation

Consider this poor code:

int *p = new int(42);
cout << "Before: " << *p << '\n';
delete p;
cout << "After:  " << *p << '\n';

Before: 42
After:  4138

Nothing mysterious here. p was a valid pointer, then it became invalid.
Just because a pointer exists, doesn’t mean that it points to something that you may access.
I know the address of the White House, but they won’t let me move in.

More Invalidation

Another way to invalidate a pointer:

string *p;
{
    string name = "John Jacob Jingleheimer Schmidt";
    p = &name;
    cout << p->size() << ' ' << *p << '\n';
}
cout << p->size() << ' ' << *p << '\n';

Iterator invalidation

vector<long> v = {253};
vector<long>::iterator it = v.begin();

cout << "Before: " << *it << '\n';

for (long i=1; i<1000; i++)
    v.push_back(i);

cout << "After:  " << *it << '\n';

Before: 253
After:  4998

Lipstick on a pig

Using auto makes the code prettier, but no better:

vector<long> v = {253};
auto it = v.begin();

cout << "Before: " << *it << '\n';

for (long i=1; i<1000; i++)
    v.push_back(i);

cout << "After: " << *it << '\n';

Before: 253
After: 7384

Reservation

Using .reserve() pre-allocates memory:

vector<long> v = {253};
v.reserve(1005);
auto it = v.begin();

cout << "Before: " << *it << '\n';

for (long i=1; i<1000; i++)
    v.push_back(i);

cout << "After:  " << *it << '\n';

Before: 253
After:  253

How often?

How often does re-allocation happen? We can find out, for any particular implemention:

vector<int> v;

for (int i=1; i<=1000; i++) {
    auto before = v.capacity();
    v.push_back(i);
    auto after = v.capacity();
    if (before != after)
        cout << i << ' ' << after << '\n';
}

How often?

Similarly, because a std::string is not much more than a vector<char>:

string s;

cout << s.size() << ' ' << s.capacity() << '\n';
for (int i=1; i<10000; i++) {
    auto before = s.capacity();
    s += 'x';
    auto after = s.capacity();
    if (before != after)
        cout << s.size() << ' ' << after << '\n';
}

Curious String Behavior

for (string s; s.size()<50; s+="abc")
    cout << sizeof(s) << ' ' << s.size() << ' '
         << &s << ' ' << (void *) &s[0] << '\n';

32 0 0x7ffdb67034e0 0x7ffdb67034f0
32 3 0x7ffdb67034e0 0x7ffdb67034f0
32 6 0x7ffdb67034e0 0x7ffdb67034f0
32 9 0x7ffdb67034e0 0x7ffdb67034f0
32 12 0x7ffdb67034e0 0x7ffdb67034f0
32 15 0x7ffdb67034e0 0x7ffdb67034f0
32 18 0x7ffdb67034e0 0x1df82c0
32 21 0x7ffdb67034e0 0x1df82c0
32 24 0x7ffdb67034e0 0x1df82c0
32 27 0x7ffdb67034e0 0x1df82c0
32 30 0x7ffdb67034e0 0x1df82c0
32 33 0x7ffdb67034e0 0x1df82f0
32 36 0x7ffdb67034e0 0x1df82f0
32 39 0x7ffdb67034e0 0x1df82f0
32 42 0x7ffdb67034e0 0x1df82f0
32 45 0x7ffdb67034e0 0x1df82f0
32 48 0x7ffdb67034e0 0x1df82f0

Order Calculation

All that reallocation looks bad.
However, science is better than first impressions.
What is the big-O time cost for adding n items to a vector?

Big-O

Let’s push 0…77 onto a vector:
- Push 0: grow to size=1, copy nothing
- Push 1: grow to size=2, copy 1 int
- Push 2: grow to size=4, copy 2 ints
- Push 4: grow to size=8, copy 4 ints
- Push 8: grow to size=16, copy 8 ints
- Push 16: grow to size=32, copy 16 ints
- Push 32: grow to size=64, copy 32 ints
- Push 64: grow to size=128, copy 64 ints
That’s 1+2+4+8+16+32+64 copies for 78 ints, 127÷78≈1.6 copies/int. O(1)!

Does it scale?

vector<char> v;
int copies = 0, iterations = 1000000;

for (int i=0; i<iterations; i++) {
    auto before = v.capacity();
    v.push_back(i);
    if (before != v.capacity())
        copies += before;
}

cout << double(copies)/iterations;

1.04858

Pre-allocation helps

Again, if we know how many items we’re going to add, we can .reserve() the space:

vector<int> v;
v.reserve(900);

cout << v.size() << ' ' << v.capacity() << '\n';
for (int i=1; i<1000; i++) {
    auto before = v.capacity();
    v.push_back(i);
    auto after = v.capacity();
    if (before != after)
        cout << v.size() << ' ' << after << '\n';
}

0 900
901 1800

Half-open interval

.begin() returns an iterator that “points” to the first element
.end() returns an iterator that “points” one past the last element
They form a half-open interval.
Remember, iterators are not pointers.

`.begin()` and `.end()`

vector<int> v = {1, 1, 2, 3, 5, 8, 13, 21, 34};
for (vector<int>::iterator it = v.begin(); it != v.end(); ++it)
    cout << *it << ' ';

1 1 2 3 5 8 13 21 34

or:

vector<int> v = {1, 1, 2, 3, 5, 8, 13, 21, 34};
for (auto it = v.begin(); it != v.end(); ++it)
    cout << *it << ' ';

1 1 2 3 5 8 13 21 34

++it is probably faster than it++.

`const`

Why does this fail?

class Foo {
    int sum() const {
        int total = 0;
        for (vector<int>::iterator it = data.begin(); it != data.end(); ++it)
            total += *it;
        return total;
    }
    vector<int> data;
};

c.cc:4: error: conversion from '__normal_iterator<const int*,[...]>' to 
   non-scalar type '__normal_iterator<int*,[...]>' requested

`const`

.sum() is a const method.
Therefore, inside .sum(), data is a const vector<int>.
- Let that sink in for a moment.
- How is a const method implemented, anyway?
- It’s not that the compiler checks for changes to member data inside a const method.
- Instead, the compiler simply regards all data members as const when compiling a const method.
Therefore, data.begin() and data.end() return iterators of type const_iterator, not type iterator.
- They’re overloaded methods!
You can’t assign const_iterator to iterator.

Restate the problem

This is the same problem:

const vector<int> v = {1, 1, 2, 3, 5, 8, 13, 21, 34};
for (vector<int>::iterator it = v.begin(); it != v.end(); ++it)
    cout << *it << ' ';

c.cc:2: error: conversion from '__normal_iterator<const int*,[...]>' to 
   non-scalar type '__normal_iterator<int*,[...]>' requested

Solution #1

One solution—use the correct type:

const vector<int> v = {1, 1, 2, 3, 5, 8, 13, 21, 34};
for (vector<int>::const_iterator it = v.begin(); it != v.end(); ++it)
    cout << *it << ' ';

1 1 2 3 5 8 13 21 34

Solution #2

A better solution—let auto figure it out:

const vector<int> v = {1, 1, 2, 3, 5, 8, 13, 21, 34};
for (auto it = v.begin(); it != v.end(); ++it)
    cout << *it << ' ';

1 1 2 3 5 8 13 21 34

Solution #3

The best solution—avoid all of this:

const vector<int> v = {1, 1, 2, 3, 5, 8, 13, 21, 34};
for (auto val : v)
    cout << val << ' ';

1 1 2 3 5 8 13 21 34

`.cbegin()` and `.cend()`

.begin() is an overloaded method. It returns a const_iterator if the object is const, iterator otherwise.
.end() is an overloaded method. It returns a const_iterator if the object is const, iterator otherwise.
.cbegin(): like .begin(), but always returns const_iterator

.cend(): like .end(), but always returns const_iterator

vector<int> v = {1, 1, 2, 3, 5, 8, 13, 21, 34};
for (auto it = v.cbegin(); it != v.cend(); ++it)
    cout << *it << ' ';

1 1 2 3 5 8 13 21 34

`.rbegin()` and `.rend()`

.rbegin(): like .begin(), but goes the other way
.rend(): like .end(), but goes the other way
.crbegin(): like .cbegin(), but goes the other way

.crend(): like .cend(), but goes the other way

array<int, 9> v = {1, 1, 2, 3, 5, 8, 13, 21, 34};
for (auto it = v.rbegin(); it != v.rend(); ++it)
    cout << *it << ' ';

34 21 13 8 5 3 2 1 1

list<int> v = {1, 1, 2, 3, 5, 8, 13, 21, 34};
for (auto it = v.crbegin(); it != v.crend(); ++it)
    cout << *it << ' ';

34 21 13 8 5 3 2 1 1

Don’t get too excited

Not all containers have the reversed versions.
- For example, forward_list, as a singly-linked list, doesn’t have reverse iterators.
For some containers, iterator and const_iterator are the same.
- A set has inherently-constant iterators, because it’s in order. If you could change the values via iterators, then it wouldn’t be in order any more.
- An unordered_set, a hash, has its own special order, and changing the values in place would be bad.

Plain old data types:

How about old C-style data?

int a[] = {1, 1, 2, 3, 5, 8, 13, 21, 34};
for (auto it = a.begin(); it != a.end(); ++it)
    cout << *it << ' ';

c.cc:2: error: request for member 'begin' in 'a', which is of non-class type 
   'int [9]'

That failed miserably. Perhaps it’s because arrays are not objects.

Free functions

Fortunately, there exist begin() and end() functions, which work for C arrays or STL containers.

begin(con) is defined as, conceptually:

    if con is a C-style array
    then
        con
    else
        con.begin()

Similarly, end(con) is:

    if con is a C-style array
    then
        con+sizeof(con)/sizeof(con[0])
    else
        con.end()

Free functions

int a[] = {1, 1, 2, 3, 5, 8, 13, 21, 34};
for (auto it = begin(a); it != end(a); ++it)
    cout << *it << ' ';

1 1 2 3 5 8 13 21 34

These work for objects, as well:

deque<int> s = {1, 1, 2, 3, 5, 8, 13, 21, 34};
for (auto it = begin(s); it != end(s); ++it)
    cout << *it << ' ';

1 1 2 3 5 8 13 21 34

What use is that? Generality for the sake of generality?

for loop

Consider any for-loop:

forward_list<int> fl = {1, 1, 2, 3, 5, 8, 13, 21, 34};
for (double v : fl)
    cout << v << ' ';

1 1 2 3 5 8 13 21 34

The compiler turns this into (approximately):

forward_list<int> fl = {1, 1, 2, 3, 5, 8, 13, 21, 34};
for (auto it = begin(fl); it != end(fl); ++it) {
    double v = *it;
    cout << v << ' ';
}

1 1 2 3 5 8 13 21 34

Which works for any container type, even C-style arrays.

CS253: Software Development with C++

Fall 2018

Iterators

CS253 Iterators

Array Traversal

Array Traversal via Pointers

vector traversal

Methods

auto is your friend

for loop

Variations

Other containers

iterator type

Finding out

Finding out

Iterator classifications

begin() & end()

.front() & .back()

Comparisons, part one

Comparisons, part two

Constructors

begin() and end() functions

Invalidation

More Invalidation

Iterator invalidation

Lipstick on a pig

Reservation

How often?

How often?

Curious String Behavior

Order Calculation

Big-O

Does it scale?

Pre-allocation helps

Half-open interval

.begin() and .end()

const

const

Restate the problem

Solution #1

Solution #2

Solution #3

.cbegin() and .cend()

.rbegin() and .rend()

Don’t get too excited

Plain old data types:

Free functions

Free functions

for loop

`vector` traversal

`auto` is your friend

`for` loop

`iterator` type

`begin()` & `end()`

`.front()` & `.back()`

`begin()` and `end()` functions

`.begin()` and `.end()`

`const`

`const`

`.cbegin()` and `.cend()`

`.rbegin()` and `.rend()`