Open and pipe

[Gaal Yahas]

Here's a basic proposal for the open and pipe builtins. It was discussed
on #perl6 today and seemed okay to the people there. I'd like to hear
your comments, since the internals side of much of this is ready and is
looking for an interface.

module Prelude-0.0.1;

class IO;

has Handle $:read_h;
has Handle $:write_h;
has Handle $:err_h;
has Process $:process_h;

multi sub open (
Str ?$filename = $?CALLER::_,
Str ?$mode = '<',
Str ?$encoding = 'auto') returns IO;

multi sub pipe (
Str ?$command = $?CALLER::_,
Str ?$encoding = 'auto',
+?to, +?from, +?skip_shell) returns IO;


Notes / questions:

Perl5-style open ">file" is no more.[1] An earlier discussion about
open here brought up URI open, but I think there was no final word,
and it looks to me that the basic open should always mean file open,
unless explicity modified. E.g., open could be used as a facade for pipe,

open 'ls', '|-'; # or even
open 'ls', :pipe => 'from'

This pipe as it is can automatically do open2/3 if both :from and :to
are given. I don't know if this is a good thing.

Is it sane to have handle members when half of the time most of them
are undef? (E.g. in ro open)

One suggestion for the 'auto' encoding was to honor a BOM if it is
present but default to ASCII otherwise. Or should we UTF-8 otherwise?
Or what's in the environment...?

And what about IO layers? :)

What else belongs here? dirhandles, sockets? How are they specced? I
was looking for a happy medium between overdwim and underlying library
primitives level.


[1] Should this be "Perl(..5) style"?

[Juerd]

Post by Gaal Yahas
open 'ls', '|-'; # or even
open 'ls', :pipe => 'from'

I dislike the hard-to-tell-apart symbols '<' and '>' for modes. 'r' and
'w' are much easier, and get rid of the awful left/right "mnemonic" that
fails to make sense to GUI users. Spelling out 'write' and 'pipe' is
even better, but sucks for oneliners and quick hacks. I already
suggested a syntax like '+$write|w' for having multiple ways to say the
same thing. I don't like an explicit :mode. Let Perl figure that out
based on passed named arguments.

But, if we're keeping these forms, then I think we should break
tradition for the pipe. In Perl, we have called pipes '==>' and '<==',
and I wonder why it would be much different for open. It also frees us
from '|-' and '-|'. It took years before I knew which one to use without
using the dictionary. Just use '==>' and '<=='.

OTOH - people might think the '<' and '>' are also related to their
operators. I see this as another good reason to not use \W symbols at
all.

Post by Gaal Yahas
This pipe as it is can automatically do open2/3 if both :from and :to
are given. I don't know if this is a good thing.

I think it's a good thing. It won't remove the traps, but it will make
using bidirectional pipes much easier - an operator that can be done
without the manual.

Post by Gaal Yahas
One suggestion for the 'auto' encoding was to honor a BOM if it is
present but default to ASCII otherwise. Or should we UTF-8 otherwise?
Or what's in the environment...?

Or guess it, -T alike. Browsers do a great (but of course not perfect)
job at guessing. I want Perl to have such dwimmery too.

Post by Gaal Yahas
And what about IO layers? :)

I can imagine a similarity to sub wrapping: $fh.wrap(Class)


Juerd

[Mark Reed]

Post by Juerd

Post by Gaal Yahas
open 'ls', '|-'; # or even
open 'ls', :pipe => 'from'

I dislike the hard-to-tell-apart symbols '<' and '>' for modes. 'r' and
'w' are much easier, and get rid of the awful left/right "mnemonic" that
fails to make sense to GUI users.

Holy matter of opinion, Batman. Œ<Œ and Œ>¹ are much easier to tell apart
than Œr¹ and Œw¹;
Œr¹ and Œw¹ make me stop and think about how you spell Œread¹ and Œwrite¹,
whereas Œ<Œ and Œ>¹ make instant visual sense, which should be appreciated
by GUI users of all people.

Left-to-right is hardly a mnemonic; you¹re writing in a language which
parses left to right, because it was created by English-speakers, and
English is written left to right. Since you pretty much have to learn
English to learn Perl, I don¹t think this is too much of a hardship, even if
it¹s counterintuitive to native speakers of Semitic languages.

And since when is Perl targeting GUI users? It¹s a PROGRAMMING LANGUAGE.
Even the original Mac developers used a command-line interface when writing
the code.

[Mark Reed]

I take some of that back ­ actually, left-to-right directionality has almost
nothing to do with understanding the < and > symbols. The arrow points in
the direction information is flowing, which is left-to-right for > but
right-to-left for <. I mean, ³>filename² is pointing at the file, so the
information is flowing into the file; ³<filename² is pointing away from the
file, so the info is flowing out of it. I don¹t see how it could be any
clearer than that.

Post by Mark Reed

Post by Juerd

Post by Gaal Yahas
open 'ls', '|-'; # or even
open 'ls', :pipe => 'from'

I dislike the hard-to-tell-apart symbols '<' and '>' for modes. 'r' and
'w' are much easier, and get rid of the awful left/right "mnemonic" that
fails to make sense to GUI users.

Holy matter of opinion, Batman. Œ<Œ and Œ>¹ are much easier to tell apart
than Œr¹ and Œw¹;
Œr¹ and Œw¹ make me stop and think about how you spell Œread¹ and Œwrite¹,
whereas Œ<Œ and Œ>¹ make instant visual sense, which should be appreciated by
GUI users of all people.
Left-to-right is hardly a mnemonic; you¹re writing in a language which parses
left to right, because it was created by English-speakers, and English is
written left to right. Since you pretty much have to learn English to learn
Perl, I don¹t think this is too much of a hardship, even if it¹s
counterintuitive to native speakers of Semitic languages.
And since when is Perl targeting GUI users? It¹s a PROGRAMMING LANGUAGE.
Even the original Mac developers used a command-line interface when writing
the code.

[Juerd]

Post by Mark Reed
Holy matter of opinion, Batman. Œ<Œ and Œ>¹ are much easier to tell apart
than Œr¹ and Œw¹;

Obviously we disagree.

What are the characters around the code supposed to be, by the way? Your
mailer tells my mailer that you're sending iso-8859-1, but I seriously
doubt that.

Post by Mark Reed
Œr¹ and Œw¹ make me stop and think about how you spell Œread¹ and Œwrite¹,
whereas Œ<Œ and Œ>¹ make instant visual sense, which should be appreciated
by GUI users of all people.

Good, now see:

my $fooh = open 'foo', '<';
my $barh = open 'bar', '>';

my $fooh = open 'foo', :r;
my $barh = open 'bar', :w;

Post by Mark Reed
<><>><<>

<
<
<
<
or the equivalent:

wrwrwwrrw
r
w
r
w
w
r
r
w

or another equivalent, which in my opinion is clearer than any of the
alternatives:

my $fooh = open 'foo', :read;
my $barh = open 'bar', :write;

write read write read write write read read write
read
write
read
write
write
read
read
write

I really do find it hard to believe that you find < and > visually more
distinctive than r and w. And is read-write <> in \W-ish? I read that as
!= or <ARGV>.

But of course, I also find it hard to believe someone's actually using
Microsoft software to send something to a geek mailinglist. ;)

Post by Mark Reed
Left-to-right is hardly a mnemonic; you¹re writing in a language which
parses left to right, because it was created by English-speakers, and
English is written left to right.

It is indeed hardly a mnemonic. Hence my use of quotes around that word.

There is nothing on the left side of my computer or screen that
indicates reading, or anything on the right side that indicates writing.

Worse, if you write

open '/etc/passwd', '<';

that visually parses as "to /etc/passwd", because that's where the
"arrow" is pointing. But does swapping the arguments make sense? It's
in that order in Perl 5, but in Perl 6, I think we shouldn't want a
text-justification like spreading of arguments over parameters, or to
make the mode argument mandatory.

Post by Mark Reed
And since when is Perl targeting GUI users? It¹s a PROGRAMMING LANGUAGE.

PROGRAMMING LANGUAGE doesn't translate to LANGUAGE USED BY PEOPLE WHO
ALL USE COMMAND LINE UTILITIES ALL THE TIME. In fact, many people use
graphical IDEs and don't like the command line. They don't magically
associate < with reading and > with writing. If there's any association
at all, my guess is that it has to do with comparison or angle brackets.


Juerd

[Mark Reed]

Post by Juerd
What are the characters around the code supposed to be, by the way? Your
mailer tells my mailer that you're sending iso-8859-1, but I seriously
doubt that.

Argh. Bad Entourage, no biscuit. Back to Mail as soon as I get Tiger
installed, I think.

Post by Juerd
But of course, I also find it hard to believe someone's actually using
Microsoft software to send something to a geek mailinglist. ;)

I had good reasons for switching to it at one time. Honest! Besides, it¹s
a work thing.

Post by Juerd
PROGRAMMING LANGUAGE doesn't translate to LANGUAGE USED BY PEOPLE WHO
ALL USE COMMAND LINE UTILITIES ALL THE TIME. In fact, many people use
graphical IDEs and don't like the command line.

I have nothing against IDEs; IDEA is pretty nice for Java coding, for
instance. But to date most Perl programmers don¹t program Perl inside an
IDE, although it would be nice if Perl 6 had an IDE coming out of the box.

My point was simply that people who program are in general more technically
savvy than those who don¹t. That doesn¹t mean they use the command line
exclusively, or at all, but it does mean that they should be capable of
understanding visual metaphors other than those of the GUI world.

[Aaron Sherman]

Post by Juerd

Post by Gaal Yahas
open 'ls', '|-'; # or even
open 'ls', :pipe => 'from'

I dislike the hard-to-tell-apart symbols '<' and '>' for modes. 'r' and
'w' are much easier, and get rid of the awful left/right "mnemonic" that
fails to make sense to GUI users.

Holy matter of opinion, Batman. Œ<Œ and Œ>¹ are much easier to tell apart
than Œr¹ and Œw¹;

As far as matter of opinion... yeah, I'm seeing a lot of that recently.

I would expect "open" to be a bit of an anachronism in P6, but still
used fairly often. For the most part, I would expect that:

my IO $read_fh = '/some/path' => 'r'; # Get an IO::File (is IO)
my IO $write_fh = '/other/path' => '>'; # Get IO::File
my IO $pipe_fh = 'program args' => $IO::ReadPipe; # Get IO::Pipe
my IO $sock_fh = 'http://www.perl.org/' => $IO::URI; # Get IO::Socket

would just DWIM. But, perhaps I'm expecting too much...

[Larry Wall]

On Wed, May 04, 2005 at 08:47:17AM -0400, Aaron Sherman wrote:
: I would expect "open" to be a bit of an anachronism in P6, but still
: used fairly often. For the most part, I would expect that:
:
: my IO $read_fh = '/some/path' => 'r'; # Get an IO::File (is IO)
: my IO $write_fh = '/other/path' => '>'; # Get IO::File
: my IO $pipe_fh = 'program args' => $IO::ReadPipe; # Get IO::Pipe
: my IO $sock_fh = 'http://www.perl.org/' => $IO::URI; # Get IO::Socket
:
: would just DWIM. But, perhaps I'm expecting too much...

Um, yes.

Larry

[Gaal Yahas]

Post by Aaron Sherman
I would expect "open" to be a bit of an anachronism in P6, but still
my IO $read_fh = '/some/path' => 'r'; # Get an IO::File (is IO)
my IO $write_fh = '/other/path' => '>'; # Get IO::File
my IO $pipe_fh = 'program args' => $IO::ReadPipe; # Get IO::Pipe
my IO $sock_fh = 'http://www.perl.org/' => $IO::URI; # Get IO::Socket
would just DWIM. But, perhaps I'm expecting too much...

Ah yes, that's another thing I was wondering about: what does opening a
pipe return. If it's a one-way pipe, okay, this may be a single handle;
but for bidirectional opens, we need $in, $out, and $err handles; and
even in the simple unidirectional case, where does the Process handle
go?[1]

In the glorious OOP version of these builtins, IO can encasulate the
three handles, plus the process handles. But if we are to provide a
pipe1 builtin, it can return a single handle (and not the pid, unless
we return a list or use OUT variables (yech)). Should we provide further
procedural interfaces for pipe2 and pipe3?

[1] "Process handle" encapsulates unix pid / win32 process handle.

[Juerd]

Post by Gaal Yahas
Ah yes, that's another thing I was wondering about: what does opening a
pipe return. If it's a one-way pipe, okay, this may be a single handle;
but for bidirectional opens, we need $in, $out, and $err handles; and

That'd be tridirectional, then.

A normal filehandle can already handle bidirection.

I think the following solution suffices in a clean way:

$h = open a pipe;

Now,
$h.in;
$h.out;
$h.err;

$h.print("foo"); # use $h.out
$l = $h.readline; # use $h.in


Juerd

[Rob Kinyon]

Would that mean that a filehandle opened readonly would throw an
exception if you attempted to either print or warn on it?

Post by Juerd

Post by Gaal Yahas
Ah yes, that's another thing I was wondering about: what does opening a
pipe return. If it's a one-way pipe, okay, this may be a single handle;
but for bidirectional opens, we need $in, $out, and $err handles; and

That'd be tridirectional, then.
A normal filehandle can already handle bidirection.
$h = open a pipe;
Now,
$h.in;
$h.out;
$h.err;
$h.print("foo"); # use $h.out
$l = $h.readline; # use $h.in
Juerd
--
http://convolution.nl/maak_juerd_blij.html
http://convolution.nl/make_juerd_happy.html
http://convolution.nl/gajigu_juerd_n.html

[Juerd]

Post by Rob Kinyon
Would that mean that a filehandle opened readonly would throw an
exception if you attempted to either print or warn on it?

I don't know what warning on a filehandle should be or do, but ignoring
that bit, yes, an exception would be the right thing to have.

Compare this to Perl 5 and see how similar it is.


Juerd

[Rob Kinyon]

Post by Juerd

Post by Rob Kinyon
Would that mean that a filehandle opened readonly would throw an
exception if you attempted to either print or warn on it?

I don't know what warning on a filehandle should be or do, but ignoring
that bit, yes, an exception would be the right thing to have.
Compare this to Perl 5 and see how similar it is.

The idea is that
$h.print() goes to $h.out
$h.readline() goes to $h.in
$h.warn() goes to $h.err

Making the tri-directional trifecta complete.

Rob

[Juerd]

Post by Rob Kinyon
$h.print() goes to $h.out
$h.readline() goes to $h.in
$h.warn() goes to $h.err
Making the tri-directional trifecta complete.

It's sort-of consistent, but I don't like it, because warnings are much
more complicated than just things that are printed to stderr.


Juerd

[Uri Guttman]

Post by Rob Kinyon
$h.print() goes to $h.out
$h.readline() goes to $h.in
$h.warn() goes to $h.err
Making the tri-directional trifecta complete.

J> It's sort-of consistent, but I don't like it, because warnings are much
J> more complicated than just things that are printed to stderr.

also you have the .err backwards. in the case of running a subprocess,
the err handle is readonly (the subprocess writes to ITS stderr with
warn). so you need another method to make a easy read of the err handle.

and you need ways to get at the handles themselves so you can use
sysread/write, select/poll and event loops. but i like the idea of
returning a single smart object which returns reasonable things in
appropriate contexts.

remember at all times when discussing i/o for p6 that the goals are to
support all forms of i/o in a consistant manner. it is much more than
just a polymorphic open or io::all. they only address one aspect of i/o
(initiating an i/o request) and that skips read/write/fcntl/event loops
and more.

uri

[Gaal Yahas]

Post by Juerd

Post by Gaal Yahas
Ah yes, that's another thing I was wondering about: what does opening a
pipe return. If it's a one-way pipe, okay, this may be a single handle;
but for bidirectional opens, we need $in, $out, and $err handles; and

That'd be tridirectional, then.

Bi and up.

Post by Juerd
A normal filehandle can already handle bidirection.

Yes, but to exploit that would be something of a perversion: the in and
out handles are very separate as far as everybody else is concerned.

Post by Juerd
$h = open a pipe;
Now,
$h.in;
$h.out;
$h.err;
$h.print("foo"); # use $h.out
$l = $h.readline; # use $h.in

Yes, if $h is the not-very-primitive version of IO. Surely the type of
$h.in is not the same as $h itself?

[Juerd]

Post by Gaal Yahas
Yes, if $h is the not-very-primitive version of IO. Surely the type of
$h.in is not the same as $h itself?

Why not? $h does IO::Handle::Tridirectional, and $h.in does not, even though
$h and $h.in are-a IO::Handle.

Or whatever the classes will be, of course.


Juerd

[Gaal Yahas]

I already suggested a syntax like '+$write|w' for having multiple
ways to say the same thing. I don't like an explicit :mode. Let
Perl figure that out based on passed named arguments.

I'd like to see this specced. What you're suggesting is that sub
signatures have something as powerful as a good getopt library; things
that spring to mind at this prospect are:

* canonical representations (eg, :w in your example should probably set
$write)

* mutually exclusive options (for open modes, :write should exclude
:append)

* computed options (if by "let Perl figure that out" $mode you didn't mean
"let perl's open figure it out with explicit code).

How would you suggest formalizing these?

(By the way, we need a spec for perl's command line, too. So far we've
been emulating perl5's in an ad-hoc manner.)

[Juerd]

Post by Gaal Yahas
* canonical representations (eg, :w in your example should probably set
$write)

Or, possibly, $w := $write.

Post by Gaal Yahas
* mutually exclusive options (for open modes, :write should exclude
:append)

I don't really care if this goes in the signature. Runtime checking is
good enough for this.

sub foo (+$write, +$append) {
fail "..." if $write and $append;
...
}

Although I don't think this particular check is needed at all, as append
isn't quite opening the file read-only. Just ignore that :write was also
given.

Post by Gaal Yahas
* computed options (if by "let Perl figure that out" $mode you didn't mean
"let perl's open figure it out with explicit code).

I did mean let open figure it out. That is, let &open decide based on
the named arguments it gets how to open something. Let *it* find a way
to specify 'a+' and stuff like that, so we can write it readably.


Juerd

[Larry Wall]

On Mon, May 02, 2005 at 10:25:08PM +0300, Gaal Yahas wrote:
: Here's a basic proposal for the open and pipe builtins. It was discussed
: on #perl6 today and seemed okay to the people there. I'd like to hear
: your comments, since the internals side of much of this is ready and is
: looking for an interface.
:
: module Prelude-0.0.1;
:
: class IO;
:
: has Handle $:read_h;
: has Handle $:write_h;
: has Handle $:err_h;
: has Process $:process_h;
:
: multi sub open (
: Str ?$filename = $?CALLER::_,
: Str ?$mode = '<',
: Str ?$encoding = 'auto') returns IO;
:
: multi sub pipe (
: Str ?$command = $?CALLER::_,
: Str ?$encoding = 'auto',
: +?to, +?from, +?skip_shell) returns IO;

I think I'd rather see something like:

multi sub open (
Str +$mode = 'r',
Str +$encoding = 'auto',
Str *@args) returns IO;

multi sub openuri (
Str +$mode = 'r',
Str +$encoding = 'auto',
Str *@args) returns IO;

multi sub openpipe (
Str +$mode = 'r',
Str +$encoding = 'auto',
Str *@args) returns IO;

multi sub openshell (
Str +$mode = 'r',
Str +$encoding = 'auto',
Str *@args) returns IO;

I don't think the command should default to $_, and I'd rather have pipes
use a list for input of the args. If we make open consistent with
that then we can open a list of files without having to clobber @ARGS.

: Notes / questions:
:
: Perl5-style open ">file" is no more.[1] An earlier discussion about
: open here brought up URI open, but I think there was no final word,
: and it looks to me that the basic open should always mean file open,
: unless explicity modified. E.g., open could be used as a facade for pipe,
:
: open 'ls', '|-'; # or even
: open 'ls', :pipe => 'from'
:
: This pipe as it is can automatically do open2/3 if both :from and :to
: are given. I don't know if this is a good thing.

I think that's just

openpipe 'ls';

By the way, P5's pipe() should probably be renamed to syspipe() or some
such to avoid confusion.

: Is it sane to have handle members when half of the time most of them
: are undef? (E.g. in ro open)

Fine by me. I'm not fond of the deep class hierarchies some languages
put onto handles. On the other hand, with roles we might be able
to compose a shallow hierarchy, and that would be okay.

: One suggestion for the 'auto' encoding was to honor a BOM if it is
: present but default to ASCII otherwise. Or should we UTF-8 otherwise?

If it's a choice between those two, definitely UTF-8, but I think we
can be smarter than that. We can pretty easily autodetect text in
UTF-16 even without a BOM, as long as we don't also have to autodetect
some legacy character set. I'd love to render BOMs obsolete, at
least on input. ("Be lenient on what you accept, and strict on what
you produce.")

: Or what's in the environment...?

More useful on output than input, I think, though it could be helpful
in telling the autodetector to look for a particular legacy character
set in addition to the UTFs.

I have another thought on that, which is that you often want a particular
output file to have the same format as some other input file. So maybe
we could have some kind of openfilter() that specifies both an input
and output file on the same (bidirectional) handle, with the assumption
that the output file will keep the format of the input file.

: And what about IO layers? :)

Yum. :-)

: What else belongs here? dirhandles, sockets? How are they specced?

Maybe something consistent like:

multi sub opendir (
Str +$mode = 'r',
Str +$encoding = 'auto',
Str *@args) returns IO;

multi sub opensocket (
Str +$mode = 'rw',
Str +$encoding = 'auto',
Str *@args) returns IO;

: I was looking for a happy medium between overdwim and underlying library
: primitives level.

We should probably reserve io() for the overdwim and sysopen() for the
underdwim, and then not sweat the middle so much. :-)

: [1] Should this be "Perl(..5) style"?

I think that'd be "Perl-{1..5} style", as it currently stands, and
assuming you want to use the "use" syntax. Also, we haven't specced
a unary .. yet. It only saves one character, and I think it's clearer
to force people to say 0..5 or 1..5 (or even "a".."z").

Larry

[Larry Wall]

On Mon, May 02, 2005 at 02:23:36PM -0700, Larry Wall wrote:
: : [1] Should this be "Perl(..5) style"?
:
: I think that'd be "Perl-{1..5} style", as it currently stands, and
: assuming you want to use the "use" syntax. Also, we haven't specced

Er, make that Perl-(1..5) instead. One week in Russia and my brain
is rotting. Couldn't have been the vodka, since I didn't have any...

Larry

[Juerd]

Post by Gaal Yahas
multi sub open (
multi sub openuri (
multi sub openpipe (
multi sub openshell (

Starting to look a lot like PHP there.

How about

&open ::= &File::open
&URI::open
&Sys::Pipe::open

And put the other aliases in the module that CGI.pm-:standard-ishly
pollutes the main namespace?

Post by Gaal Yahas
I don't think the command should default to $_

Eh...

Why?!


Juerd

[Larry Wall]

On Tue, May 03, 2005 at 12:32:58AM +0200, Juerd wrote:
: Larry Wall skribis 2005-05-02 14:23 (-0700):
: > multi sub open (
: > multi sub openuri (
: > multi sub openpipe (
: > multi sub openshell (
:
: Starting to look a lot like PHP there.

And I care about that because PHP is such an unsuccessful language? :-)

I'd just like all the opens to sort to the same place in the manual.

: How about
:
: &open ::= &File::open
: &URI::open
: &Sys::Pipe::open
:
: And put the other aliases in the module that CGI.pm-:standard-ishly
: pollutes the main namespace?

I'm not too worried about polluting the main namespace, now that
it's equivalent to Perl 5's third class keywords. I think that Perl 6
should end up with *more* names in the global namespace than Perl 5 had,
as long as we don't end up with as many inconsistent names as PHP.

: > I don't think the command should default to $_
:
: Eh...
:
: Why?!

Because $_ is primarily for the use of inner loops, not outer loops,
and open tends to be in the outer loop rather than the inner loop.
And because if there is no inner loop we're doing some kind of slurp
or io() pipe that bypasses open to begin with. And partly because of
the mess in Perl 5 from 1-arg open, though that's probably irrational.

Larry

[Larry Wall]

On Mon, May 02, 2005 at 03:53:42PM -0700, Larry Wall wrote:
: On Tue, May 03, 2005 at 12:32:58AM +0200, Juerd wrote:
: : How about
: :
: : &open ::= &File::open
: : &URI::open
: : &Sys::Pipe::open
: :
: : And put the other aliases in the module that CGI.pm-:standard-ishly
: : pollutes the main namespace?
:
: I'm not too worried about polluting the main namespace, now that
: it's equivalent to Perl 5's third class keywords. I think that Perl 6
: should end up with *more* names in the global namespace than Perl 5 had,
: as long as we don't end up with as many inconsistent names as PHP.

On the other hand, I *do* think these should live in their own packages
and be aliased into the global namespace. Perhaps generic open could
just search a path for handlers as in the original RFC:

use open :path<URI File>;

But in general I wouldn't mind seeing things like:

$handle = open URI: $uri;
$handle = Pipe.open([$prog, @args], :rw);

Larry

[Juerd]

Post by Larry Wall
: Starting to look a lot like PHP there.
And I care about that because PHP is such an unsuccessful language? :-)
I'd just like all the opens to sort to the same place in the manual.

One of my biggest problems with PHP is that it puts in the name of
a function information that IMO belongs in an argument or namespace.

In the case of open, I think namespaces make the most sense. Even though
they all return a handle, the way that handle is created is indeed
radically different. And important here, the way they can fail is also
very different.

A good example of how PHP differs from Perl in this respect is sorting.
Perl provides one sort and that single sort is capable of any kind of
sorting. PHP on the other hand has 16 different *sort functions in the
standard language (core plus extensions in the standard distribution,
because extensions are compiled in).

More examples can be found at <http://tnx.nl/php>.

I don't mind having numerous aliases to harder to find neatly
categorized functions, as long as there is a single pragma to avoid it.

Post by Larry Wall
: > I don't think the command should default to $_
: Why?!
Because $_ is primarily for the use of inner loops, not outer loops,
and open tends to be in the outer loop rather than the inner loop.

As someone who tries to write clean code, I agree that the outer loop
should use a full name for the loop variable.

But as someone who writes oneliners every day, I disagree that any such
decision of how I should code should be made by the language designers.

$_ exists at many levels, and I frankly don't see a big problem with

for @files {
open;
say uc for =$_;
close;
}

(ignoring at this point that open accepts a list of handles. I'm sure
there are situations, especially in oneliners, where that isn't the best
solution.)

Post by Larry Wall
And partly because of the mess in Perl 5 from 1-arg open, though
that's probably irrational.

It is indeed.


Juerd

[Luke Palmer]

Post by Juerd

Post by Larry Wall
: > I don't think the command should default to $_
: Why?!
Because $_ is primarily for the use of inner loops, not outer loops,
and open tends to be in the outer loop rather than the inner loop.

As someone who tries to write clean code, I agree that the outer loop
should use a full name for the loop variable.
But as someone who writes oneliners every day, I disagree that any such
decision of how I should code should be made by the language designers.
$_ exists at many levels, and I frankly don't see a big problem with
open;
say uc for =$_;
close;
}

Wait, so you want open to both open the filename in $_ /and/ set $_ to
the opened filehandle? Also, I don't buy your argument, since you can
do:

for @files {
say uc for =io($_);
}

Which the oneliner side of you should find awful nice.

Luke

[Juerd]

Post by Luke Palmer
Wait, so you want open to both open the filename in $_ /and/ set $_ to
the opened filehandle?

You are right. The example code makes no sense at all.

Post by Luke Palmer
say uc for =io($_);

I didn't know io was blessed already.

Post by Luke Palmer
Which the oneliner side of you should find awful nice.

It does indeed!


Juerd

[Larry Wall]

On Tue, May 03, 2005 at 11:22:06AM +0200, Juerd wrote:
: I didn't know io was blessed already.

It pretty much is, just not the subsequent overloading of > and <.
We'll use ==> and <== instead.

Larry

[Uri Guttman]

LW> multi sub opensocket (
LW> Str +$mode = 'rw',
LW> Str +$encoding = 'auto',
LW> Str *@args) returns IO;

and how will that support async (non-blocking) connects? or listen
sockets? i assume this is sorta the replacement for IO::Socket which is
a pure key/value api (except for the shortcut single arg host:port
which is nice to also support). for async you need to provide a callback
in the signature (beyond all the other socket args), an optional timeout
(with its own callback). the callbacks could be defaulted to subs in the
current module/namespace but the coder may also want an OO callback
which needs an object and method. another little optional argument is a
private data field. sure an object callback could handle on a per object
basis. but what if one object wanted to manage multiple async socket
connections? each one needs to be identified in the callbacks which is
what the private data is for. it is passed as an arg to the callback
sub/method.

LW> We should probably reserve io() for the overdwim and sysopen() for the
LW> underdwim, and then not sweat the middle so much. :-)

then io would be common blocking connects (which is what most socket
connections are) and sysopen is for sockets????. why not just support
the standard socket subs as in perl5? they could be in a module but they
are just simple wrappers around the system calls.

i would prefer if opensocket's signature were fully fleshed out with
named args (with some defaults). just passing in an extra list is a poor
api as there are so many socket options. just read PBP (when it comes
out) for damian's take on long arg lists. :)

uri

[Matt Fowles]

All~

Post by Uri Guttman
LW> multi sub opensocket (
LW> Str +$mode = 'rw',
LW> Str +$encoding = 'auto',
and how will that support async (non-blocking) connects? or listen
sockets? i assume this is sorta the replacement for IO::Socket which is
a pure key/value api (except for the shortcut single arg host:port
which is nice to also support). for async you need to provide a callback
in the signature (beyond all the other socket args), an optional timeout
(with its own callback). the callbacks could be defaulted to subs in the
current module/namespace but the coder may also want an OO callback
which needs an object and method. another little optional argument is a
private data field. sure an object callback could handle on a per object
basis. but what if one object wanted to manage multiple async socket
connections? each one needs to be identified in the callbacks which is
what the private data is for. it is passed as an arg to the callback
sub/method.

Currying obviates the need for everything but a sub callback. If you
want a callback to a method, curry the object. If you want private
data, curry the data. After you are done currying you will have a
simple sub to pass in as the callback, the peasants rejoice, and
libraries will have a simpler interface.

Matt

[Uri Guttman]

MF> All~
LW> multi sub opensocket (
LW> Str +$mode = 'rw',
LW> Str +$encoding = 'auto',

Post by Uri Guttman
and how will that support async (non-blocking) connects? or listen
sockets? i assume this is sorta the replacement for IO::Socket which is
a pure key/value api (except for the shortcut single arg host:port
which is nice to also support). for async you need to provide a callback
in the signature (beyond all the other socket args), an optional timeout
(with its own callback). the callbacks could be defaulted to subs in the
current module/namespace but the coder may also want an OO callback
which needs an object and method. another little optional argument is a
private data field. sure an object callback could handle on a per object
basis. but what if one object wanted to manage multiple async socket
connections? each one needs to be identified in the callbacks which is
what the private data is for. it is passed as an arg to the callback
sub/method.

MF> Currying obviates the need for everything but a sub callback. If you
MF> want a callback to a method, curry the object. If you want private
MF> data, curry the data. After you are done currying you will have a
MF> simple sub to pass in as the callback, the peasants rejoice, and
MF> libraries will have a simpler interface.

i like the concept but how would that work without actually creating
closures? to convert a sub call to a method on an object is code, not
just presetting args (which is what i gather currying is). now just
using a closure would work but i am not in favor of allocating them just
for this when other optional args can do the job. even with the full
interface as i want it, you can use closures as your callbacks and keep
it shorter. i just don't see the win of more code vs more args.

uri

[Matt Fowles]

All~

Post by Uri Guttman
MF> All~
LW> multi sub opensocket (
LW> Str +$mode = 'rw',
LW> Str +$encoding = 'auto',

Post by Uri Guttman
and how will that support async (non-blocking) connects? or listen
sockets? i assume this is sorta the replacement for IO::Socket which is
a pure key/value api (except for the shortcut single arg host:port
which is nice to also support). for async you need to provide a callback
in the signature (beyond all the other socket args), an optional timeout
(with its own callback). the callbacks could be defaulted to subs in the
current module/namespace but the coder may also want an OO callback
which needs an object and method. another little optional argument is a
private data field. sure an object callback could handle on a per object
basis. but what if one object wanted to manage multiple async socket
connections? each one needs to be identified in the callbacks which is
what the private data is for. it is passed as an arg to the callback
sub/method.

MF> Currying obviates the need for everything but a sub callback. If you
MF> want a callback to a method, curry the object. If you want private
MF> data, curry the data. After you are done currying you will have a
MF> simple sub to pass in as the callback, the peasants rejoice, and
MF> libraries will have a simpler interface.
i like the concept but how would that work without actually creating
closures? to convert a sub call to a method on an object is code, not
just presetting args (which is what i gather currying is). now just
using a closure would work but i am not in favor of allocating them just
for this when other optional args can do the job. even with the full
interface as i want it, you can use closures as your callbacks and keep
it shorter. i just don't see the win of more code vs more args.

Curried functions are most likely implemented as closures internally.
But perl6 has C< .assuming > which does most of the ugly syntactic
work of creating the closure for us. As for the cost of allocating
the closure, I tend not to worry about that till I know it is a
problem. Most callbacks are created once and called many times, so
the allocation costs are not a great concern.

On the other hand, I view a simpler interface to a library as a big
win. Yes, optional arguments could be added which allow for such
things, but then the library has to document and manage all of these
arguments, and the library user has to learn them all.

Matt

[Uri Guttman]

MF> Currying obviates the need for everything but a sub callback. If you
MF> want a callback to a method, curry the object. If you want private
MF> data, curry the data. After you are done currying you will have a
MF> simple sub to pass in as the callback, the peasants rejoice, and
MF> libraries will have a simpler interface.

Post by Uri Guttman
i like the concept but how would that work without actually creating
closures? to convert a sub call to a method on an object is code, not
just presetting args (which is what i gather currying is). now just
using a closure would work but i am not in favor of allocating them just
for this when other optional args can do the job. even with the full
interface as i want it, you can use closures as your callbacks and keep
it shorter. i just don't see the win of more code vs more args.

MF> Curried functions are most likely implemented as closures internally.
MF> But perl6 has C< .assuming > which does most of the ugly syntactic
MF> work of creating the closure for us. As for the cost of allocating
MF> the closure, I tend not to worry about that till I know it is a
MF> problem. Most callbacks are created once and called many times, so
MF> the allocation costs are not a great concern.

i was more curious about the syntax of a sub that curries a method call
on an object. it seems to me that you need actual code to call the
method. the currying only presets args (or clear me up on that). there
is no way for the args to magically assemble themselves into a method
call on that object without some perl code. that makes it a real closure
and not something as simple as currying.

MF> On the other hand, I view a simpler interface to a library as a big
MF> win. Yes, optional arguments could be added which allow for such
MF> things, but then the library has to document and manage all of these
MF> arguments, and the library user has to learn them all.

documenting the ways to create needed closures seems to me to be more
complex than just documenting more arguments. there is no new concept to
teach (few even understand or use closures in p5) nor new syntax to
describe.

even if you reduce the main callback to a single arg, it needs a name as
it is optional and you save only one named arg (either the object or
method). what about the optional timeout callback? in most cases (at
least how i do it) that callback wants to use the same object as the
main callback. so this would mean two closure args. in the common case
of fully named args, you could have the method/sub names defaulted (say
to socket_connected and connection_timed_out) and then all you need is
the single object argument. so that is fewer args to pass and to know
for the common case.

and this gets you a nice way to differentiate between sync and async
open_socket calls. if it has an object arg (could be a class name too),
then it does callbacks to that object which makes it async. it could
handle a string as a sub name (symrefs lurk here) or a CODE ref. the
internals can figure that out easily now.

<ODD ASYNC RANT>
a particular odd variation is to let real sync calls act like async
calls. this is so you can write a module with socket calls and i/o and
have it run sync in some cases (e.g. single scripts) or async in others
(long lived demons). so the code is the same but with callbacks in all
socket calls. some other vaguely global option will enable all those
calls to be async in reality if desired. since the callbacks control the
behavior, this means the same module can be used in both ways. this is
how i wish many of cpan's socket modules were written. if they
completely isolated requesting i/o from handling the results, then it
would be easy to use them in sync or async style.
</ODD ASYNC RANT>

anyhow, we can't keep arguing about arguments. tomato/tomahto and all
that. and there will always be modules to change the api to your own
favorite api style. my main point was that socket calls do have many
more args than plain open and having them in a large slurpy list is
bad. they need to be properly named (but optional and with some useful
defaults). exactly how they are named and organized is less important to
me than having all the needed options (like callbacks) in there.

and this brings me back to another point about how async stuff will
behave in a normal op loop (and not an event loop). i have yet to see a
solution to this (nor barely seen it discussed). e.g. an async
(non-blocking) socket connect is made. a callback maybe done later. what
is the code doing now? how will the mainline code know when the callback
is done? it can poll some data (or a semaphore or whatever) that the
callback sets but that is wasteful. the other classic solution is to
provide a wait call (waiting on again some data or semaphore or what)
that internally does a blocking call to select/poll/whatever. this can
also have a timeout which can be used too. i think a 'wait' func should
be in the language as well as a basic event loop type of thing. they
should be standard (at worst core modules) so we don't get different
incompatible event loops and related stuff.

'nuff for now.

uri

[Larry Wall]

On Mon, May 02, 2005 at 10:51:57PM -0400, Uri Guttman wrote:
: >>>>> "LW" == Larry Wall <***@wall.org> writes:
:
: LW> multi sub opensocket (
: LW> Str +$mode = 'rw',
: LW> Str +$encoding = 'auto',
: LW> Str *@args) returns IO;
:
: and how will that support async (non-blocking) connects? or listen
: sockets? i assume this is sorta the replacement for IO::Socket which is
: a pure key/value api (except for the shortcut single arg host:port
: which is nice to also support).

Well, we could go that way, and just say that

multi sub openfoo (*@args) {...}

gets all of its parameters, named or otherwise, from the slurpy array
just like Perl 5. But then MMD has little to work with. So you're
probably right, and there need to be more named arguments. I was
mostly just trying to keep *@args out there at the end so that
openpipe could have multiple arguments, but we could just do that
with a single [...] parameter in any event, which could by default
be a positional parameter at the front, but also be moved to
the end as a named parameter if it has too much endweight.

Larry

[Aaron Sherman]

Post by Uri Guttman
LW> multi sub opensocket (
LW> Str +$mode = 'rw',
LW> Str +$encoding = 'auto',
and how will that support async (non-blocking) connects? or listen
sockets?

This is why named aliases for constructors are a bad idea. Nice theory,
but bad idea.

Unless the language allows us to specify that a sub IS an alias for a
constructor, e.g.:

sub opensocket := IO::Socket.new;

Why? Because IO::Socket.new takes parameters that are built out of its
entire inheritance tree, so a change to IO::Handle might radically
modify the signature of the constructor.

[Uri Guttman]

LW> multi sub opensocket (
LW> Str +$mode = 'rw',
LW> Str +$encoding = 'auto',

Post by Uri Guttman
and how will that support async (non-blocking) connects? or listen
sockets?

AS> This is why named aliases for constructors are a bad idea. Nice theory,
AS> but bad idea.

i am in agreement there. but i am advocating for a proper set of args
for socket connections regardless of the name of the sub/method. it
could even be in io() provided there is a way to note it is a socket
connect/listen and also pass it named args. larry already agreed with
the named arguments point.

AS> Unless the language allows us to specify that a sub IS an alias for a
AS> constructor, e.g.:

AS> sub opensocket := IO::Socket.new;

AS> Why? Because IO::Socket.new takes parameters that are built out of its
AS> entire inheritance tree, so a change to IO::Handle might radically
AS> modify the signature of the constructor.

makes sense. we should look at the p5 IO:: tree and see what we want to
salvage/savage from it as well as io::all. each has its good and bad
points and hopefully we can figure out which is which. :)

uri

[Graham Barr]

Post by Uri Guttman
AS> Why? Because IO::Socket.new takes parameters that are built out of its
AS> entire inheritance tree, so a change to IO::Handle might
radically
AS> modify the signature of the constructor.
makes sense. we should look at the p5 IO:: tree and see what we want to
salvage/savage from it as well as io::all. each has its good and bad
points and hopefully we can figure out which is which. :)

Yes. p5 IO:: has evolved a bit over time and some things have turned
out to be bad choices in hindsight.

For example, one thing I am still not convinced was the right thing was
to create a hierarchy such that a given class only had methods that
were valid for it. For example the recent issue on p5p with seek etc
not being in IO::Handle but IO::Seekable and the core using IO::Handle.

Also the way IO::Socket::* was done instead of all sockets created by
IO::Socket->new

these are all things that should be reconsidered and I am sure in all
cases there will be people on both sides of the fence

Graham.