Re: Request for opinions: bug vs feature - change intokenization of yield from

From: Date: Sat, 20 Jul 2024 16:04:52 +0000
Subject: Re: Request for opinions: bug vs feature - change intokenization of yield from
References: 1 2 3 4 5 6 7  Groups: php.internals 
Request: Send a blank email to [email protected] to get a copy of this message
Hi

On 7/20/24 17:42, Juliette Reinders Folmer wrote:
I already mentioned cast tokens before. Whitespace is perfectly acceptable within the parentheses of these. Comments are not: https://3v4l.org/A6Sgj and https://3v4l.org/nE9H8 Now you may argue that cast tokens "feel like" a single operator, but that's subjective and there's even a sniff to enforce no spacing within cast parentheses as apparently people do pad them with spaces - and doing so is allowed in PHP. * _* Qualifier: spaces and tabs are allowed inside cast parentheses, but new lines are not..._
I stand corrected. Though that indeed is another odd special case with not allowing newlines. Looking at the lexer, there's another case where tabs and spaces may appear, but not other whitespace. Between '<<<' and the delimiter of a heredoc.
So to get back to my original point, as of PHP 8.3 is the **only** token which allows for a comment to be tokenized as part of the token. There is no other token which allows that.
PHP users have no idea what a token is internally. I'm looking at this from a PHP user perspective. It looks like two keywords, it walks like two keywords and it quacks like two keywords. I find it reasonable for users to consider this as two keywords and not care about how it's implemented internally.
So, what "feels" natural (whitespace-wise) to one person may not be the same for the next, but comments _within_ tokens is different thing and should in my opinion, not be allowed.
As I've said: I agree that the current situation is unfortunate. But the correct solution is not "disallow comments", but "split the T_YIELD_FROM into T_YIELD T_WHITESPACE T_FROM_FOR_YIELD_FROM". Best regards Tim Düsterhus

Thread (32 messages)

« previous php.internals (#124518) next »