instance method

Prism::Source#code_units_offset

code_units_offset(byte_offset, encoding)

lib/prism/parse_result.rb View on GitHub ↗

# File tmp/rubies/ruby-4.0.0/lib/prism/parse_result.rb, line 124
def code_units_offset(byte_offset, encoding)
  byteslice = (source.byteslice(0, byte_offset) or raise).encode(encoding, invalid: :replace, undef: :replace)

  if encoding == Encoding::UTF_16LE || encoding == Encoding::UTF_16BE
    byteslice.bytesize / 2
  else
    byteslice.length
  end
end

Returns the offset from the start of the file for the given byte offset counting in code units for the given encoding.

This method is tested with UTF-8, UTF-16, and UTF-32. If there is the concept of code units that differs from the number of characters in other encodings, it is not captured here.

We purposefully replace invalid and undefined characters with replacement characters in this conversion. This happens for two reasons. First, it’s possible that the given byte offset will not occur on a character boundary. Second, it’s possible that the source code will contain a character that has no equivalent in the given encoding.

See also #code_units_column #column #code_units_cache #deep_freeze also in 3.4

‹ prev#code_units_column next ›#column