in reply to Re: using stdin with sqlldr
in thread using stdin with sqlldr

because we typically insert about a million records per load. Doing this with an insert statement would take weeks or months.

Replies are listed 'Best First'.
Re^3: using stdin with sqlldr
by cmdrake (Acolyte) on Nov 18, 2008 at 22:23 UTC
    I tried the "documented" ways to get STDIN working, but it failed. So, I tried named pipes, and got the working on Windows. You could do something very similar on *nixy platforms with mknod/mkfifo
    Control file:
    LOAD DATA INFILE '\\.\pipe\sql_pipe\' REPLACE INTO TABLE my_table FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' (col1,col2,col3,col4)
    Perl:
    use strict; use Win32::Pipe; $|=1; # autoflush for messages my $pipe_name = "sql_pipe"; my $pipe = new Win32::Pipe($pipe_name)|| die "Can't Create Named Pipe: + $!\n"; open(my $ldr,"|sqlldr.exe a/b\@c control=control.ctl log") || die "Unable to execute sqlldr: $!"; $pipe->Connect(); #... do something interesting to get data, and probably loop $pipe->Write(join(",",$1,$2,$3,$4)."\n"); #... $pipe->Disconnect(); $pipe->Close(); close $ldr;
    Note: the pipe in the open was just a leftover from my first attempt... You could remove that and do something more sensible for launching sqlldr...
      Hi, thanks for the response. This helps!
      Do you know of anyway to load a clob file into sqlldr WITHOUT designating the clob as a filename/file pointer???
      In all the examples sqlldr uses a filename or pointer to a file to load clobs. I would like to pass my clob data as a string of text like I do with all the other fields. Do you think it would be possible to pipe it to the clob field, kinda like the way you piped each row to sqlldr in your example above. See I'm trying to avoid writing any data to disk, so I want to avoid writing all my data to a file just so I can load it as a clob field in sqlldr.
      I tried this and got it working in just three lines:
      open (my $sqlldr, "|sqlldr usr/pass\@db control=sbteststream.ctl log=sb.log bad=sb.bad direct=true")
      || die "Cannot open sqlldr\n";
      print $sqlldr "16107364Ø454370Ø60Ø10eb2b53ea80d0b63e72d8a63e6e1b75Ø3800Ø\n";
      close($sqlldr);
Re^3: using stdin with sqlldr
by cmdrake (Acolyte) on Nov 18, 2008 at 19:29 UTC