yes: support invalid unicode on Windows

[oech3]

One of a common usecase of yes is generating periodic binary data which is blocked by

coreutils/src/uu/yes/src/yes.rs

Lines 73 to 82 in 1717352

	// But, on Windows, we must hop through a String.
	#[cfg(not(any(unix, target_os = "wasi")))]
	{
	for part in itertools::intersperse(i.map(|a| a.to_str()), Some(" ")) {
	let bytes = part
	.ok_or_else(|| USimpleError::new(1, translate!("yes-error-invalid-utf8")))?
	.as_bytes();
	buf.extend_from_slice(bytes);
	}
	}

currently.

[xtqqczze]

But does Windows stdio support non-UTF-8 byte sequences? Maybe not in console mode?

[oech3]

I think stdout converted to File can do that as same as dd.

[xtqqczze]

I think stdout converted to File can do that as same as dd.

Uh right, well I guess we can't do that unless the console's code page is set to UTF-8.

[xtqqczze]

Related:

• Reading from and writing to the Windows console should be lossy rust-lang/rust#116871
• Windows: Don't error on broken non UTF-8 output rust-lang/rust#134534

[oech3]

If yes > file generates valid binary. It is useful enough.

[xtqqczze]

So we just need to careful if we are in console mode with the code page set to UTF-16.

[ChrisDenton]

My suggestion would be to handle the simpleyes > file case first and leave the harder console case until later. A Windows console can be detected by calling GetConsoleMode on a handle and seeing if it succeeds.

Though tbh I don't understand the utility of outputting binary data directly to any console (without piping through something else to make it human-readable). While it would be nice to have full compatibility, I don't think it's a big priority here.

[oech3]

I have no access on Windows now. But I see

+ target/debug/cat.exe target/debug/cat.exe
D:\a\_temp\cbe04bb7-3d2e-4997-8597-ae3aab286872.sh: line 4: target/debug/cat.exe: No such file or directory

on the CI. Assuming binary input is invalid unicode.

[oech3]

Cannot we remove

coreutils/src/uu/yes/src/yes.rs

Line 57 in 1717352

#[allow(clippy::unnecessary_wraps, reason = "needed on some platforms")]

due to support of utf16?