Maintenance Ruby 3.4 gets fixes, but no new features. Switch to Ruby 4.0 →
instance method
RubyVM::AbstractSyntaxTree::Node#all_tokens
() -> ::Array[[ ::Integer, ::Symbol, ::String, [ ::Integer, ::Integer, ::Integer, ::Integer ] ]]?
node.all_tokens → array
ast.rb
View on GitHub ↗
# File tmp/rubies/ruby-3.4.1/ast.rb, line 205
def all_tokens
Primitive.ast_node_all_tokens
end
Returns all tokens for the input script regardless the receiver node. Returns nil if keep_tokens is not enabled when parse method is called.
Example
root = RubyVM::AbstractSyntaxTree.parse("x = 1 + 2", keep_tokens: true) root.all_tokens # => [[0, :tIDENTIFIER, "x", [1, 0, 1, 1]], [1, :tSP, " ", [1, 1, 1, 2]], ...] root.children[-1].all_tokens # => [[0, :tIDENTIFIER, "x", [1, 0, 1, 1]], [1, :tSP, " ", [1, 1, 1, 2]], ...]