Re: Request for opinions: bug vs feature -changeintokenization of yield from

From: Date: Tue, 23 Jul 2024 10:16:20 +0000
Subject: Re: Request for opinions: bug vs feature -changeintokenization of yield from
References: 1 2 3 4 5 6 7 8 9 10 11 12  Groups: php.internals 
Request: Send a blank email to [email protected] to get a copy of this message
On 21.07.2024 at 13:28, Tim Düsterhus wrote:

> On 7/20/24 20:31, Christoph M. Becker wrote:
>
>> If the tokenizer would tokenize a whole file as a single token, would
>> that also be correct?  Of course, I'm exaggerating, but
>
> The function is documented as:
>
>> token_get_all() parses the given code string into PHP language tokens
>> using the Zend engine's lexical scanner.
>
> So if Zend engine's lexical scanner would tokenize a complete PHP file
> as a single token, then the output would be correct, yes.

Apparently we cannot even agree that there is a bug, and as such we
cannot fix it in PHP 8.3 according to the release process documentation.

>> <https://3v4l.org/qIf2c> doesn't look
>> correct to me – "yield /* comment
>> */ from" shouldn't be a single token.
>
> Looking at this from a user expectation perspective and not a technical
> perspective, neither should yield from be a single token in:
>
>     var_dump(token_get_all('<?php yield from $foo;'));

ACK

Cheers,
Christoph


Thread (32 messages)

« previous php.internals (#124555) next »