string_tokeniser Class Template Reference
[String Library]

#include <stlsoft/string/string_tokeniser.hpp>

Inheritance diagram for string_tokeniser:

List of all members.

---

Detailed Description

template<typename S, typename D, typename B = skip_blank_tokens<true>, typename V = S, typename T = string_tokeniser_type_traits<S, V>, typename P = string_tokeniser_comparator<D, S, T>>
class stlsoft::string_tokeniser< S, D, B, V, T, P >

A class template that provides string tokenising behaviour.

This class takes a string, and a delimiter, and fashions a sequence from the given string, with each element determined with respect to the delimiter

Parameters:

	S	The string type
	D	The delimiter type (can be a string type or a character type)
	B	The ignore-blanks type. Defaults to skip_blank_tokens<true>
	V	The value type (the string type that will be used for the values). Defaults to S
	T	The string type traits type. Defaults to string_tokeniser_type_traits<S, V>
	P	The tokeniser comparator type. Defaults to string_tokeniser_comparator<D, S, T>

This class template provides tokenising services of a string (of type S) with a delimiter (of type D). The four other template parameters, which are defaulted, are used for tailoring the tokenising behaviour for special uses.

The two typical supported tokenising scenarios are:

• tokenising a string with a character (e.g. '\n')
• tokenising a string with a string (e.g. "\\r\\n")

More exotic scenarios are supported by customising the comparator and type-traits parameters. (See stlsoft::charset_tokeniser.)

1. Tokenising a string with a character.

This uses a specialisation whereby the first template parameter is a string type, and the second parameter is a corresponding character type.

The following code shows a specialisation using std::string and char, and will output: abc,def,ghi,jkl,

stlsoft::string_tokeniser<std::string, char>  tokens(":abc::def:ghi:jkl::::::::::", ':');

std::copy(tokens.begin(), tokens.end(), std::ostream_iterator<std::string>(std::cout, ","));

The following code shows a specialisation using stlsoft::basic_simple_string<wchar_t> and wchar_t, and will output: abc-def-ghi-jkl-

typedef stlsoft::basic_simple_string<wchar_t>   string_t;
string_t                                        s(L"|abc||def|ghi|jkl||||||||||");
stlsoft::string_tokeniser<string_t, wchar_t>    tokens(s, L'|');

std::copy(tokens.begin(), tokens.end(), std::ostream_iterator<string_t, wchar_t>(std::wcout, L"-"));

Optionally, you can stipulate that the blanks be retained by specifying the third template parameter as skip_blank_tokens<false>, as in the following, which will output: ,abc,,def,ghi,jkl,,,,,,,,,,

stlsoft::string_tokeniser<  std::string
                        ,   char
                        ,   stlsoft::skip_blank_tokens<false>
                        >                tokens(":abc::def:ghi:jkl::::::::::", ':');

std::copy(tokens.begin(), tokens.end(), std::ostream_iterator<std::string>(std::cout, ","));

Note:

The tokeniser uses String Access Shims to elicit the string from the given type, so any type that for which shims are defined can be passed to the constructor, as in the following, which will output: abc;def;ghi;jkl;

#include <stlsoft/string/string_tokeniser.hpp>
#include <winstl/shims/access/string.hpp>

#include <iostream>
#include <iterator>

int main()
{
  HWND  hwndButton = ::CreateWindowEx(0, "BUTTON", "+abc++def+ghi+jkl++++++++++", 0, 0, 0, 0, 0, NULL, (HMENU)0, NULL, NULL);

  stlsoft::string_tokeniser<  std::string
                          ,   char
                          ,   stlsoft::skip_blank_tokens<true>
                          >                tokens(hwndButton, '+');
  std::copy(tokens.begin(), tokens.end(), std::ostream_iterator<std::string>(std::cout, ";"));
  return 0;
}

2. Tokenising a string with a string.

This uses a specialisation whereby the first template parameter is a string type, and the second parameter is a corresponding string type.

The following code shows a specialisation using std::string and std::string, and will output: abc,def,ghi,jkl,

stlsoft::string_tokeniser<std::string, std::string>  tokens("\r\nabc\r\n\r\ndef\r\nghi\r\njkl\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n", "\r\n");

std::copy(tokens.begin(), tokens.end(), std::ostream_iterator<std::string>(std::cout, ","));

Examples:

by_library/windows_registry/enum_values/enum_values.cpp.

Public Types
Member Types
typedef string_tokeniser< S, D, B, V, T, P >	class_type
	The current parameterisation of the type.
typedef string_tokeniser< S, D, B, V, T, P >	tokeniser_type
	This tokeniser parameterisation.
typedef S	string_type
	The sequence string type.
typedef D	delimiter_type
	The delimiter type.
typedef B	blanks_policy_type
	The blanks policy type.
typedef V	value_type
	The value type.
typedef T	traits_type
	The traits type.
typedef P	comparator_type
	The tokeniser comparator type.
typedef traits_type::value_type	char_type
	The character type.
typedef size_t	size_type
	The size type.
typedef ss_ptrdiff_t	difference_type
	The difference type.
typedef const value_type	const_reference
	The non-mutating (const) reference type.
Public Member Functions
Construction
	string_tokeniser (char_type const *psz, delimiter_type const &delim)
	Tokenise the given C-string with the given delimiter.
template<typename S1>
	string_tokeniser (S1 const &str, delimiter_type const &delim)
	Tokenise the given string with the given delimiter.
	string_tokeniser (char_type const *psz, size_type cch, delimiter_type const &delim)
	Tokenise the specified length of the given string with the given delimiter.
template<typename I>
	string_tokeniser (I from, I to, delimiter_type const &delim)
	Tokenise the given range with the given delimiter.
Iteration
const_iterator	begin () const
	Begins the iteration.
const_iterator	end () const
	Ends the iteration.
Attributes
bool	empty () const
	Indicates whether the search sequence is empty.
Classes
class	const_iterator
	Iterator for string_tokeniser, supporting the Forward Iterator concept. More...

---

Member Typedef Documentation

typedef string_tokeniser<S, D, B, V, T, P> class_type

The current parameterisation of the type.

Reimplemented in charset_tokeniser.

typedef string_tokeniser<S, D, B, V, T, P> tokeniser_type

This tokeniser parameterisation.

typedef S string_type

The sequence string type.

Reimplemented in charset_tokeniser.

typedef D delimiter_type

The delimiter type.

Reimplemented in charset_tokeniser.

typedef B blanks_policy_type

The blanks policy type.

Reimplemented in charset_tokeniser.

typedef V value_type

The value type.

Reimplemented in charset_tokeniser.

typedef T traits_type

The traits type.

Reimplemented in charset_tokeniser.

typedef P comparator_type

The tokeniser comparator type.

Reimplemented in charset_tokeniser.

typedef traits_type::value_type char_type

The character type.

Reimplemented in charset_tokeniser.

typedef size_t size_type

The size type.

Note:

This no longer relies on a size_type member type of the traits type (T). It is defined as size_t

Reimplemented in charset_tokeniser.

typedef ss_ptrdiff_t difference_type

The difference type.

Note:

This no longer relies on a difference_type member type of the traits type (T). It is defined as ptrdiff_t

typedef const value_type const_reference

The non-mutating (const) reference type.

Reimplemented in charset_tokeniser.

---

Constructor & Destructor Documentation

string_tokeniser	(	char_type const *	psz,
		delimiter_type const &	delim
	)			[inline]

Tokenise the given C-string with the given delimiter.

Parameters:

	psz	Pointer to C-string whose contents will be tokenised
	delim	The delimiter to perform the tokenisation

Note:

The tokeniser class takes a copy of psz. It does not alter the contents of psz

string_tokeniser	(	S1 const &	str,
		delimiter_type const &	delim
	)			[inline]

Tokenise the given string with the given delimiter.

Parameters:

	str	The string whose contents will be tokenised
	delim	The delimiter to perform the tokenisation

Note:

The tokeniser class takes a copy of str. It does not alter the contents of str

string_tokeniser	(	char_type const *	psz,
		size_type	cch,
		delimiter_type const &	delim
	)			[inline]

Tokenise the specified length of the given string with the given delimiter.

Parameters:

	psz	Pointer to C-string whose contents will be tokenised
	cch	The number of characters in psz to use
	delim	The delimiter to perform the tokenisation

Note:

The tokeniser class takes a copy of psz. It does not alter the contents of psz

string_tokeniser	(	I	from,
		I	to,
		delimiter_type const &	delim
	)			[inline]

Tokenise the given range with the given delimiter.

Parameters:

	from	The start of the asymmetric range to tokenise
	to	The start of the asymmetric range to tokenise
	delim	The delimiter to use

---

Member Function Documentation

const_iterator begin

(

)

const [inline]

Begins the iteration.

Returns:

An iterator representing the start of the sequence

const_iterator end

(

)

const [inline]

Ends the iteration.

Returns:

An iterator representing the end of the sequence

bool empty

(

)

const [inline]

Indicates whether the search sequence is empty.

---

The documentation for this class was generated from the following file:

• stlsoft/string/string_tokeniser.hpp