ne_token, ne_qtoken — string tokenizers


#include <ne_string.h>
char *ne_token(char **str,
 char sep);
char *ne_qtoken(char **str,
 char sep,
 const char *quotes);


ne_token and ne_qtoken tokenize the string at the location stored in the pointer str. Each time the function is called, it returns the next token, and modifies the str pointer to point to the remainer of the string, or NULL if there are no more tokens in the string. A token is delimited by the separator character sep; if ne_qtoken is used any quoted segments of the string are skipped when searching for a separator. A quoted segment is enclosed in a pair of one of the characters given in the quotes string.

The string being tokenized is modified each time the tokenizing function is called; replacing the next separator character with a NUL terminator.


The following function prints out each token in a comma-separated string list, which is modified in-place:

static void splitter(char *list)
  do {
    printf("Token: %s\n", ne_token(&list, ','));
  while (list);