tokenizer.html 7.7 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122
  1. <div id="documentation" class="span9">
  2. <div class="classContent">
  3. <div class="membersBackground"></div>
  4. <div class=" members pos0">
  5. <div class=" membersContent pos0">
  6. <h1 class="memberHeader"><span class="name">Tokenizer</span>
  7. </h1>
  8. <ul data-tabs="tabs" class="nav tabs pos0">
  9. <li data-dropdown="dropdown" class="dropdown">
  10. <ul class="dropdown-menu">
  11. </ul>
  12. </li>
  13. <li data-dropdown="dropdown" class="dropdown"><a href="#" data-toggle="dropdown" class="dropdown-toggle"><span>Constructors (1)</span><b class="caret"></b></a>
  14. <ul class="dropdown-menu">
  15. <li id="dropdown_Tokenizer.new" data-id="Tokenizer.new" class="memberLink"><a href="#Tokenizer.new" class="" title="Tokenizer.new (constructor)" data-id="Tokenizer.new">new</a>
  16. </li>
  17. </ul>
  18. </li>
  19. <li data-dropdown="dropdown" class="dropdown"><a href="#" data-toggle="dropdown" class="dropdown-toggle"><span>Functions (1)</span><b class="caret"></b></a>
  20. <ul class="dropdown-menu">
  21. <li id="dropdown_Tokenizer.getLineTokens" data-id="Tokenizer.getLineTokens" class="memberLink"><a href="#Tokenizer.getLineTokens" class="" title="Tokenizer.getLineTokens (class method)" data-id="Tokenizer.getLineTokens">getLineTokens</a>
  22. </li>
  23. </ul>
  24. </li>
  25. <li data-dropdown="dropdown" class="dropdown">
  26. <ul class="dropdown-menu">
  27. </ul>
  28. </li>
  29. <li data-dropdown="dropdown" class="dropdown">
  30. <ul class="dropdown-menu">
  31. </ul>
  32. </li>
  33. <li data-dropdown="dropdown" class="dropdown">
  34. <ul class="dropdown-menu">
  35. </ul>
  36. </li>
  37. <li data-dropdown="dropdown" class="dropdown">
  38. <ul class="dropdown-menu">
  39. </ul>
  40. </li>
  41. </ul>
  42. </div>
  43. </div>
  44. <article id="Tokenizer" data-title="Tokenizer (class)" class="article">
  45. <div class="section description">
  46. <div class="memberContent"><p>This class takes a set of highlighting rules, and creates a tokenizer out of them. For more information, see <a href="https://github.com/ajaxorg/ace/wiki/Creating-or-Extending-an-Edit-Mode#wiki-extendingTheHighlighter">the wiki on extending highlighters</a>.</p>
  47. </div>
  48. </div>
  49. </article>
  50. <h3 class="sectionHeader">Constructors</h3>
  51. <article id="Tokenizer.new" data-title="Tokenizer.new (constructor)" class="article">
  52. <div class="section method">
  53. <div class="memberContent">
  54. <div class="title"><i id="Tokenizer.new" class="methodToggle methodClicker inactive icon-caret-right"></i>
  55. <ul class="signatures">
  56. <li class="signature">
  57. <ul>
  58. <li class="signature-call"><span class="constructorIdentifier">new </span><span id="Tokenizer.new" class="member-name methodClicker">Tokenizer</span>(<span class="sigArgList"><a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Object" class="argument methodClicker" title="Object" data-id="Object">Object</a> rules, <a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/String" class="argument methodClicker" title="String" data-id="String">String</a> flag</span>)</li>
  59. </ul>
  60. <ul class="metaInfo">
  61. </ul>
  62. </li>
  63. </ul>
  64. </div>
  65. <div class="sideToggler">
  66. <div id="ellipsis_Tokenizer.new" class="ellipsis_description"><p>Constructs a new tokenizer based on the given rules and flags.</p>
  67. </div>
  68. <div class="description"><p>Constructs a new tokenizer based on the given rules and flags.</p>
  69. <h4>Arguments</h4><table class="argumentTable argument-list table table-striped table-bordered"><tr class="argumentRow "><td class="argName ">rules</td><td class="argType" "><a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Object" class="" title="Object" data-id="Object">Object</a></td><td class="argDescription "><p>Required. The highlighting rules</p>
  70. </td></tr><tr class="argumentRow "><td class="argName ">flag</td><td class="argType" "><a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/String" class="" title="String" data-id="String">String</a></td><td class="argDescription "><p>Required. Any additional regular expression flags to pass (like &quot;i&quot; for case insensitive)</p>
  71. </td></tr></table>
  72. </div>
  73. </div>
  74. </div>
  75. </div>
  76. </article>
  77. <h3 class="sectionHeader">Methods</h3>
  78. <article id="Tokenizer.getLineTokens" data-title="Tokenizer.getLineTokens (class method)" class="article">
  79. <div class="section method">
  80. <div class="memberContent">
  81. <div class="title"><i id="Tokenizer.getLineTokens" class="methodToggle methodClicker inactive icon-caret-right"></i>
  82. <ul class="signatures">
  83. <li class="signature">
  84. <ul>
  85. <li class="signature-call"><span id="Tokenizer.getLineTokens" class="member-name methodClicker"><span class="sigClassName">Tokenizer.</span><span class="sigMemberName">getLineTokens</span></span>(<span class="sigArgList"><a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Object" class="argument methodClicker" title="Object" data-id="Object">Object</a> line, <a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Object" class="argument methodClicker" title="Object" data-id="Object">Object</a> startState</span>)
  86. <li class="signature-returns">
  87. <ul class="argument-types">
  88. <li class="argument-type"><a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Object" class="returnType " title="Object" data-id="Object">Object</a></li>
  89. </ul>
  90. </li></li>
  91. </ul>
  92. <ul class="metaInfo">
  93. </ul>
  94. </li>
  95. </ul>
  96. </div>
  97. <div class="sideToggler">
  98. <div id="ellipsis_Tokenizer.getLineTokens" class="ellipsis_description"><p>Returns an object containing two properties: <code>tokens</code>, which contains all the tokens; and <code>state</code>, the current state.</p>
  99. </div>
  100. <div class="description"><p>Returns an object containing two properties: <code>tokens</code>, which contains all the tokens; and <code>state</code>, the current state.</p>
  101. <h4>Arguments</h4><table class="argumentTable argument-list table table-striped table-bordered"><tr class="argumentRow "><td class="argName ">line</td><td class="argType" "><a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Object" class="" title="Object" data-id="Object">Object</a></td><td class="argDescription "><p>Required. </p>
  102. </td></tr><tr class="argumentRow "><td class="argName ">startState</td><td class="argType" "><a href="https://developer.mozilla.org/en-US/docs/JavaScript/Reference/Global_Objects/Object" class="" title="Object" data-id="Object">Object</a></td><td class="argDescription "><p>Required. </p>
  103. </td></tr></table>
  104. </div>
  105. </div>
  106. </div>
  107. </div>
  108. </article>
  109. </div>
  110. <script defer src="./resources/javascripts/ux.js"></script>
  111. <script src="./resources/javascripts/clicker.js"></script>
  112. <script src="./resources/javascripts/jquery-scrollspy.js"></script>
  113. <script defer src="./resources/javascripts/disqus-ext.js"></script>
  114. <script defer src="./resources/javascripts/ga.js"></script>
  115. <div id="disqus_thread"></div>
  116. </div>