Class Sprout::FCSHLexer
In: bundles/as3/lib/sprout/fcsh_lexer.rb
Parent: Object

This class should allow us to parse the stream output that FCSH provides. It was largely inspired by "LittleLexer" (rubyforge.org/projects/littlelexer/) which is a beautiful and concise general purpose lexer written by John Carter. Unfortunately, LittleLexer did not support streamed input, which we definitely need.

Methods

new   scan_process   scan_stream  

Constants

PROMPT = ':prompt'
WARNING = ':warning'
ERROR = ':error'
PRELUDE = ':prelude'
PRELUDE_EXPRESSION = /(Adobe Flex Compiler.*\n.*\nCopyright.*\n)/m

Public Class methods

new(out=nil)

[Source]

# File bundles/as3/lib/sprout/fcsh_lexer.rb, line 39
    def initialize(out=nil)
      @out = out || $stdout
      @regex_to_token = [
                        [/(.*Warning:.*\^.*)\n/m,   WARNING], # Warning encountered
                        [/(.*Error:.*\^.*)\n/m,     ERROR], # Error encountered
                        [PRELUDE_EXPRESSION,        PRELUDE],
                        [/\n\(fcsh\)/,              PROMPT] # Prompt for input
                       ]
    end

Public Instance methods

scan_process(process_runner) {|token if block_given?| ...}

[Source]

# File bundles/as3/lib/sprout/fcsh_lexer.rb, line 49
    def scan_process(process_runner)
      tokens = [];
      # Collect Errors and Warnings in a way that doesn't
      # Block forever when we have none....
      t = Thread.new {
        scan_stream(process_runner.e) do |token|
          yield token if block_given?
          tokens << token
        end
      }
      
      # Collect stdout from the process:
      scan_stream(process_runner.r) do |token|
        yield token if block_given?
        tokens << token
      end
      
      process_runner.e.sync = true

      # GROSS HACK! 
      # It seems we need to wait
      # for the fsch $stderr buffer to flush?
      # There must be a better way... Anyone?
      # Should we move to Highline for interactive
      # shell applications?
      # http://rubyforge.org/projects/highline/
      # In fact - this problem actually ruins
      # the entire implementation, the larger/longer
      # it takes for errors to be bufferred, the more
      # likely it is we'll return without displaying them.
      # The only way to overcome this with the current 
      # implementation, is to increase the timeout so that
      # FCSH takes a long, long time on every compilation!!!
      sleep(0.2)
      
      t.kill
      return tokens
    end
scan_stream(reader) {|token if block_given?| ...}

We need to scan the stream as FCSH writes to it. Since FCSH is a persistent CLI application, it never sends an EOF or even a consistent EOL. In order to tokenize the output, we need to attempt to check tokens with each character added. scan_stream will block and read characters from the reader provided until it encounters a PROMPT token, at that time, it will return an array of all tokens found. It will additionally yield each token as it‘s found if a block is provided.

[Source]

# File bundles/as3/lib/sprout/fcsh_lexer.rb, line 96
    def scan_stream(reader)
      tokens = []
      partial = ''
      index = 0
      while(true) do
        code = reader.getc
        return if code.nil?
        
        partial << code.chr
        token = next_token(partial)
        if(token)
          tokens << token
          yield token if block_given?
          partial = ''
          break if(token[:name] == PROMPT || token[:name] == ERROR)
        end
      end
      
      return tokens
    end

[Validate]