CodeRay::TokenStream

TokenStream

The TokenStream class is a fake Array without elements.

It redirects the method << to a block given at creation.

This allows scanners and Encoders to use streaming (no tokens are saved, the input is highlighted the same time it is scanned) with the same code.

See CodeRay.encode_stream and CodeRay.scan_stream

Attributes

size[R]

The Array is empty, but size counts the tokens given by <<.

Public Class Methods

new(&block) click to toggle source

Creates a new TokenStream that calls block whenever its << method is called.

Example:

  require 'coderay'
  
  token_stream = CodeRay::TokenStream.new do |text, kind|
    puts 'kind: %s, text size: %d.' % [kind, text.size]
  end
  
  token_stream << ['/\d+/', :regexp]
  #-> kind: rexpexp, text size: 5.

     # File lib/coderay/tokens.rb, line 313
313:     def initialize &block
314:       raise ArgumentError, 'Block expected for streaming.' unless block
315:       @callback = block
316:       @size = 0
317:     end

Public Instance Methods

<<(token) click to toggle source

Calls block with token and increments size.

Returns self.

     # File lib/coderay/tokens.rb, line 322
322:     def << token
323:       @callback.call(*token)
324:       @size += 1
325:       self
326:     end

dump() click to toggle source

A TokenStream cannot be dumped. Use Tokens.

     # File lib/coderay/tokens.rb, line 335
335:     def dump
336:       raise NotImplementedError, 'A TokenStream cannot be dumped.'
337:     end

optimize() click to toggle source

A TokenStream cannot be optimized. Use Tokens.

     # File lib/coderay/tokens.rb, line 340
340:     def optimize
341:       raise NotImplementedError, 'A TokenStream cannot be optimized.'
342:     end

stream?() click to toggle source

Whether the object is a TokenStream.

Returns true.

     # File lib/coderay/tokens.rb, line 292
292:     def stream?
293:       true
294:     end

text_size() click to toggle source

This method is not implemented due to speed reasons. Use Tokens.

     # File lib/coderay/tokens.rb, line 329
329:     def text_size
330:       raise NotImplementedError,
331:         'This method is not implemented due to speed reasons.'
332:     end

Home Classes Methods

In Files

Parent

Methods

Files

Class Index

CodeRay::TokenStream

TokenStream

Attributes

Public Class Methods

Public Instance Methods