tokenized

tokenized_example

Syntax	Code
Pipe	rng \| boost::adaptors::tokenized(regex) rng \| boost::adaptors::tokenized(regex, i) rng \| boost::adaptors::tokenized(regex, rndRng) rng \| boost::adaptors::tokenized(regex, i, flags) rng \| boost::adaptors::tokenized(regex, rndRng, flags)
Function	boost::adaptors::tokenize(rng, regex) boost::adaptors::tokenize(rng, regex, i) boost::adaptors::tokenize(rng, regex, rndRng) boost::adaptors::tokenize(rng, regex, i, flags) boost::adaptors::tokenize(rng, regex, rndRng, flags)

Syntax

Code

Pipe

rng | boost::adaptors::tokenized(regex)
rng | boost::adaptors::tokenized(regex, i)
rng | boost::adaptors::tokenized(regex, rndRng)
rng | boost::adaptors::tokenized(regex, i, flags)
rng | boost::adaptors::tokenized(regex, rndRng, flags)

Function

boost::adaptors::tokenize(rng, regex)
boost::adaptors::tokenize(rng, regex, i)
boost::adaptors::tokenize(rng, regex, rndRng)
boost::adaptors::tokenize(rng, regex, i, flags)
boost::adaptors::tokenize(rng, regex, rndRng, flags)

Precondition:
- Let T denote typename range_value<decltype(rng)>::type, then regex has the type basic_regex<T> or is implicitly convertible to one of these types.
- i has the type int.
- the value_type of rndRng is int.
- flags has the type regex_constants::syntax_option_type.
Returns: A range whose iterators behave as if they were the original iterators wrapped in regex_token_iterator. The first iterator in the range would be constructed by forwarding all the arguments of tokenized() to the regex_token_iterator constructor.
Throws: Whatever constructing and copying equivalent regex_token_iterators might throw.
Range Category: Random Access Range
Range Return Type: boost::tokenized_range<decltype(rng)>
Returned Range Category: Random Access Range

tokenized_example

#include <boost/range/adaptor/tokenized.hpp>
#include <boost/range/algorithm/copy.hpp>
#include <boost/assign.hpp>
#include <iterator>
#include <iostream>
#include <vector>

int main(int argc, const char* argv[])
{
    using namespace boost::adaptors;

    typedef boost::sub_match< std::string::iterator > match_type;

    std::string input = " a b c d e f g hijklmnopqrstuvwxyz";
    boost::copy(
        input | tokenized(boost::regex("\\w+")),
        std::ostream_iterator<match_type>(std::cout, "\n"));

    return 0;
}

This would produce the output:

a
b
c
d
e
f
g
hijklmnopqrstuvwxyz

Boost C++ Libraries

tokenized

tokenized_example