Possible side-effects of Erlang/OTP 28.0 debug info format change

[Eiji7]

Elixir and Erlang/OTP versions

Found when migrating (by compile from source in asdf) from Erlang version 27.3.4 to 28.0.
Should affect every Elixir version (compatible with new Erlang of course) and is still part of the main tree.

Operating system

Gentoo Linux with default/linux/amd64/23.0/desktop/plasma profile and 6.12.31 kernel version.

Current behavior

I'm not sure if it's a question or a serious issue since it's at least partially related to private API, but please keep in mind:

1. I'm not asking for a support for private API
2. I'm not asking for Elixir question
3. The issue is hidden more deep in catch-all pattern

To explain the issue let me share some code to visualise it:

@spec get_types(binary()) :: list(String.t())
def get_types(binary) do
  with {:ok, {_module, [debug_info: debug_info_v1]}} <- :beam_lib.chunks(binary, [:debug_info]),
       attrs = get_attrs(debug_info_v1) do
    for {:attribute, _number, :type, type} <- attrs do
      type
      |> Typespec.type_to_quoted()
      |> Macro.to_string()
    end
  end
end

# Erlang/OTP < 28.0
@spec get_attrs({:debug_info_v1, :elixir_erl, {:elixir_v1, %{}, attrs}}) :: attrs
      when attrs: nonempty_list()
defp get_attrs({:debug_info_v1, :elixir_erl, {:elixir_v1, %{}, attrs}}), do: attrs

# Erlang/OTP 28.0 and later
@spec get_attrs(
        {:debug_info, :elixir_erl,
         {:elixir_v1, %{}, [{:debug_info, {:elixir_erl, {:elixir_v1, %{}, attrs}}} | any()]}}
      ) :: attrs
      when attrs: nonempty_list()
defp get_attrs({:debug_info_v1, _backend, {[], data}}) do
  {:debug_info, {:elixir_erl, {:elixir_v1, %{}, attrs}}} = List.keyfind(data, :debug_info, 0)
  attrs
end

Please note that I'm using this code only for tests as I'm aware that I cannot count on any support for private API - that's not a problem.

As mentioned the possible problem is hidden in private API. Please take a look at this part of linked code from main:

elixir/lib/elixir/lib/code/typespec.ex

Lines 155 to 173 in 954133d

	defp typespecs_abstract_code(module) do
	with {module, binary} <- get_module_and_beam(module),
	{:ok, {_, [debug_info: {:debug_info_v1, backend, data}]}} <-
	:beam_lib.chunks(binary, [:debug_info]) do
	case data do
	{:elixir_v1, %{}, specs} ->
	# Fast path to avoid translation to Erlang from Elixir.
	{:ok, specs}
	_ ->
	case backend.debug_info(:erlang_v1, module, data, []) do
	{:ok, abstract_code} -> {:ok, abstract_code}
	_ -> :error
	end
	end
	else
	_ -> :error
	end
	end

Expected behavior

Hopefully you already noticed the difference between my code and Elixir implementation. In my case I'm pattern-matching on the keyword value of debug_info key. However Elixir strictly depends on the "old" format. The possible problem is that the code falls back not even to catch-all pattern in case statement, but into else block of with statement resulting in :error return.

This means that in some cases the typespecs would be never returned for Erlang/OTP 28.0 which may be a serious problem. That's why I'm not sure if that's an actual issue or a question if we are aware of a new format and we don't care about catch-all pattern. So I wonder if you would like to see PR for it or the current behaviour is acceptable/expected.

This may be a serious problem since dialyxir does not see all types and functions (in all apps!). Most probably dialyzer was not updated for new debug info format, but I did not investigated it yet, so I can't say for sure. In worst case we may have more changes like this one.

Please let me know what do you think about it.

[josevalim]

Thank you for the report. I need a mechanism that reproduces this bug, as I need to understand the root cause before I give any suggestion. Perhaps the root cause is how the information is written, not how it is read.

[josevalim]

Here is an example running Elixir on Erlang/OTP 28:

iex(2)> :beam_lib.chunks(:code.which(String), [:debug_info])
{:ok,
 {String,
  [
    debug_info: {:debug_info_v1, :elixir_erl,
     {:elixir_v1,
      %{
        attributes: [],

You can see there is no nesting.

[Eiji7]

@josevalim I see … I use it in project with a lots of metaprogramming, so it's not an 5-min task and perhaps it would be easier to write it by hand, so let me describe what I'm doing:

1. I'm adding @compile debug_info: true to the module body to make sure the debug info is available in tests
2. The rest work is a rather trivial as the macro uses a simple for loop and helper function that said loop uses

for {var, spec} <- compile_time_data do
  @type unquote(var) :: unquote(spec)
end

So I guess it has to be something with @compile attribute. That's said it happens only in Erlang 28.0 - i have verified it by testing 1.18.4-otp-27 and 1.19.0-rc.0-otp-27 and both uses the old format, but 1.19.0-rc.0-otp-28 uses new one. I will try to create a minimal code for this issue asap.

[Eiji7]

I have confirmed that @compile attribute causes the issue. See minimal code:

defmodule Simple do
  @compile debug_info: true
  @type simple :: integer()
end
# {:module, _module, binary} - the binary is needed in tests (`.exs` file so no beam files)
|> elem(2)
|> :beam_lib.chunks([:debug_info])
|> IO.inspect()

When @compile attribute is removed the old format is used and it happens only on Erlang 28.0.

[josevalim]

Perfect. I will investigate it then. Btw, you don't need @compile debug_info: true, you can also set debug_info: true in test_elixirc_options: [debug_info: true].

[Eiji7]

Agree, but it would enable debug info for all modules. I enable it only for specific ones as it's needed in the tests.

[josevalim]

Oh, and this happens if you use exclusively @compile {:debug_info, true}. It works fine if you use @compile :debug_info.

[Eiji7]

Yes, I can confirm this on my machine as well. 👀