capturing system command output

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

capturing system command output

mkavanagh
Hi,

I'm using the below to successfully invoke OS commands from within a grails app:

                def rC = attrs["runCmd"]
                def procCmd = rC.execute()
                procCmd.consumeProcessOutput();
                out << procCmd.text  

NB: the last line above isn't doing anything.

How can I capture and process/view the output produced in memory by putting it into a local variable to process such as....

               def processOutpout = procCmd.?????

Once in memory I can do some processing and display in a web page if need be.

Thanks in advance.


-Mike
Reply | Threaded
Open this post in threaded view
|

Re: capturing system command output

Nathan Wells
Based on these pages:

http://groovy.codehaus.org/Process+Management
http://pleac.sourceforge.net/pleac_groovy/processmanagementetc.html
http://groovy.codehaus.org/groovy-jdk/java/lang/Process.html#consumeProcessOutput%28%29

consumeProcessOutput doesn't do what you think it does. It will, in fact, throw away all the process output. Based on the docs I read, this is how you should do what you are trying to do:

    def process = "runCmd".execute()
    process.waitFor()
    out << process.text

The call to waitFor may not be strictly necessary... the docs aren't very clear. I know that it would be necessary if you were using straight Java.

Nathan Wells


On Mon, Nov 19, 2012 at 10:27 AM, mkavanagh <[hidden email]> wrote:
Hi,

I'm using the below to successfully invoke OS commands from within a grails
app:

                def rC = attrs["runCmd"]
                def procCmd = rC.execute()
                procCmd.consumeProcessOutput();
                out << procCmd.text

NB: the last line above isn't doing anything.

How can I capture and process/view the output produced in memory by putting
it into a local variable to process such as....

               def processOutpout = procCmd.?????

Once in memory I can do some processing and display in a web page if need
be.

Thanks in advance.


-Mike



--
View this message in context: http://grails.1312388.n4.nabble.com/capturing-system-command-output-tp4637864.html
Sent from the Grails - user mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|

Re: capturing system command output

Steve Ronderos-2
I think Nathan is on the right track. Here is another resource I found that mentions reading the process response.


-Steve

On Monday, November 19, 2012, Nathan Wells wrote:
Based on these pages:

http://groovy.codehaus.org/Process+Management
http://pleac.sourceforge.net/pleac_groovy/processmanagementetc.html
http://groovy.codehaus.org/groovy-jdk/java/lang/Process.html#consumeProcessOutput%28%29

consumeProcessOutput doesn't do what you think it does. It will, in fact, throw away all the process output. Based on the docs I read, this is how you should do what you are trying to do:

    def process = "runCmd".execute()
    process.waitFor()
    out << process.text

The call to waitFor may not be strictly necessary... the docs aren't very clear. I know that it would be necessary if you were using straight Java.

Nathan Wells


On Mon, Nov 19, 2012 at 10:27 AM, mkavanagh <<a href="javascript:_e({}, &#39;cvml&#39;, &#39;mike.kavanagh@odx.uk.com&#39;);" target="_blank">mike.kavanagh@...> wrote:
Hi,

I'm using the below to successfully invoke OS commands from within a grails
app:

                def rC = attrs["runCmd"]
                def procCmd = rC.execute()
                procCmd.consumeProcessOutput();
                out << procCmd.text

NB: the last line above isn't doing anything.

How can I capture and process/view the output produced in memory by putting
it into a local variable to process such as....

               def processOutpout = procCmd.?????

Once in memory I can do some processing and display in a web page if need
be.

Thanks in advance.


-Mike



--
View this message in context: http://grails.1312388.n4.nabble.com/capturing-system-command-output-tp4637864.html
Sent from the Grails - user mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|

Re: capturing system command output

mkavanagh
In reply to this post by mkavanagh
Hi,

Thanks for your help with this - I have had some success but it is a somewhat unsatisfactory.

Here is the only way in which I have successfully captured output from an OS command to process within a  tag.

                def sOut = new StringBuilder()
                def sErr = new StringBuilder()

                def rC = attrs["runCmd"]
                def procCmd = rC.execute()
                procCmd.consumeProcessOutput(sOut, sErr)
                procCmd.waitFor()

                if (sErr) out << "Error : " + sErr + "</p>"

// Tried without success...
//
// if (sOut) out << " Output: " + sOut + "</p>"
// if (procCmd.in.text) out << " text: " + procCmd.in.text + "</p>"
// if (procCmd.text) out << " text: " + procCmd.text + "</p>"
// if (procCmd.err.text) out << " text: " + procCmd.err.text + "</p>"
               
} // end runCommand


Surprisingly only the error stream captures the output from the OS command - which seems odd.

Note the commented out process options I have tried without any success.

Incidentally, I am using  SpringSource Toolset Vers 2.9.2 on  MS Vista OS.

I intend to stick with this approach until I come up with a better solution.

Any further thoughts would be welcome.


Thanks

-Mike
Reply | Threaded
Open this post in threaded view
|

Re: capturing system command output

bksaville
I would remove the call to consumeProcessOutput.  The likely reason why your other attempts have failed is that you are calling procCmd.in.text and procCmd.err.text twice - once for the if() call, and once to actually get the output. Only the first will return anything.  Try doing it like so:

def rC = attrs["runCommand"]
def procCommand = rC.execute()
def sOut = procCommand.in.text
def sErr = procCommand.err.text
def exitCode = procCommand.exitValue() 

-Brian

On Fri, Nov 23, 2012 at 9:31 AM, mkavanagh <[hidden email]> wrote:
Hi,

Thanks for your help with this - I have had some success but it is a
somewhat unsatisfactory.

Here is the only way in which I have successfully captured output from an OS
command to process within a  tag.

                def sOut = new StringBuilder()
                def sErr = new StringBuilder()

                def rC = attrs["runCmd"]
                def procCmd = rC.execute()
                procCmd.consumeProcessOutput(sOut, sErr)
                procCmd.waitFor()

                if (sErr) out << "Error : " + sErr + "</p>"

// Tried without success...
//
//              if (sOut) out << " Output: " + sOut + "</p>"
//              if (procCmd.in.text) out << " text: " + procCmd.in.text + "</p>"
//              if (procCmd.text) out << " text: " + procCmd.text + "</p>"
//              if (procCmd.err.text) out << " text: " + procCmd.err.text + "</p>"

} // end runCommand


Surprisingly only the error stream captures the output from the OS command -
which seems odd.

Note the commented out process options I have tried without any success.

Incidentally, I am using  SpringSource Toolset Vers 2.9.2 on  MS Vista OS.

I intend to stick with this approach until I come up with a better solution.

Any further thoughts would be welcome.


Thanks

-Mike



--
View this message in context: http://grails.1312388.n4.nabble.com/capturing-system-command-output-tp4637864p4638008.html
Sent from the Grails - user mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|

Re: capturing system command output

talldave
In reply to this post by mkavanagh
i did battle with this same issue previously.  here's an example of how i got it working:

ProcessBuilder pb = new ProcessBuilder( "ls -l".split(' ') )
pb.directory( new File( "/home/david" ) )
pb.redirectErrorStream( true )

Process p = pb.start()
InputStream is = p?.inputStream

// def sb = new StringBuilder()

int i=0
while ( (i=is?.read()) != -1 ) {
  print (i as char)
  // sb.append( i as char )
}

p?.waitFor()
is?.close()
// parse sb

note that with ProcessBuilder if you want to pass parameters as part of your command it will need to be a list, hence the .split(' ').  also, inside the while loop, instead of the print statement you can append each char to a StringBuilder or StringBuffer and then parse the output in a separate thread for long running processes.

one possible downside is that the redirectErrorStream(true) will send both standard and error output to the same stream.  for my scenario this made it simpler to deal with, but if you want to parse the error stream separately you wouldn't do the redirect and instead access it via p.errorStream.

hope that helps!
Reply | Threaded
Open this post in threaded view
|

Re: capturing system command output

mkavanagh
Hi talldave,

Thanks to for everyone's help with this, all suggestions up talldave's idea of character level processing of output has not improved things - so I am keen to pursue this idea.

I have had some success with this.

If I combine the output and error stream as suggested it is fine - but for completeness I would like to split them up and process them separately.

But I am getting an java problem 'java.lang.ArrayIndexOutOfBoundsException' which I believe is a result of the streaming capture in the command process tag not working properly. I'm sure I'm doing something wrong when I try and capture both stream separately.
 
Here is the full script:

        def rC1 = attrs["runCmd"]
       
        ProcessBuilder pb = new ProcessBuilder( rC1.split(' ') )
        pb.directory( new File( "C:\\TmpHV" ) )
// pb.redirectErrorStream( true )

        Process p = pb.start()

// Capture output stream from the command
       
        def sbOut = new StringBuilder()
        InputStream is = p?.inputStream
        int i=0
        while ( (i=is?.read()) != -1 ) {
          sbOut.append( i as char )
        }

// Capture error stream from the command
       
        def sbErr = new StringBuilder()
        ErrorStream isErr = p?.errorStream
        int ie=0
        while ( (ie=isErr?.read()) != -1 ) {
          sbErr.append( ie as char )
        }

        p?.waitFor()
        is?.close()
        isErr?.close()
       
        print (sbErr) // send all errors to the console
        out << sbOut // return output to be processed

I'm sure I need to better understand what this is doing - as I'm not sure how it can capture the output stream and then the error stream.

Thanks again for your help.


-Mike
Reply | Threaded
Open this post in threaded view
|

Re: capturing system command output

talldave
i'm not able to replicate an ArrayIndexOutOfBoundsException running this in the groovy console, but i suppose i'm not too surprised that you'd have issues blocking on the output stream and then trying later to read the error stream.  i'm also not sure what the ErrorStream class is, but, i was able to get the following to work in the console.

in this scenario i removed my permissions to the file test.txt, so that triggers the error stream to be read with a "permission denied" message, and no data on the output stream.  if i instead execute the command "ls -l" i get the directory listing on the output stream, with no data on the error stream.

curious to hear if this works in your situation.


ProcessBuilder pb = new ProcessBuilder( "cat test.txt".split(' ') )
pb.directory( new File( "/tmp" ) )
//pb.redirectErrorStream( true )

Process p = pb.start()
       
def sbOut = new StringBuilder()
def sbErr = new StringBuilder()

InputStream is = p?.inputStream
InputStream isErr = p?.errorStream

int i=0
int ie=0

while ( (i=is?.read()) != -1 || (ie=isErr?.read()) != -1 ) {
  if ( i != -1 ) sbOut.append( i as char )
  if ( ie != -1 ) sbErr.append( ie as char )
}

p?.waitFor()
is?.close()
isErr?.close()

println "err = ${sbErr}"
println "out = ${sbOut}"
Reply | Threaded
Open this post in threaded view
|

Re: capturing system command output

mkavanagh
In reply to this post by mkavanagh
Hi Talldave,

Again thanks for getting back to me and correcting my syntax.

When I use this approach (managing explicitly both the output and error streams) the app hangs on the that command and fails to return.

Here is the syntax:

  def sbOut = new StringBuilder()
        def sbErr = new StringBuilder()
       
        InputStream isOut = p?.inputStream
        InputStream isErr = p?.errorStream
       
        int i=0
        int ie = 0
       
        while ( (i=isOut?.read())||(ie=isErr?.read()) != -1 ) {
        if (i != -1) sbOut.append( i as char )
        if (ie != -1) sbErr.append( ie as char )
        }

        p?.waitFor()
        isOut?.close()
        isErr?.close()
       
        print (" RunCommand1 - Out: $sbOut")
        print (" RunCommand1 - Err: $sbErr")

It's odd that my environment seems to be behaving differently from yours - (It looks like it needs some extra command to force the process to complete)?

If I just manage one of these streams it works fine. My other conclusion is that for some reason the specific OS commands I am using are only produces output data on the error stream. I am using ffmpeg commands to analyse and process images and video - namely: ffmpeg to do some processing and ffprobe to inspect files. This is confusing and difficult to put into a generic tab routine. I'm also on a windows dos environment which may have something to do with it given that ffmpeg was primarily developed in unix.

The original (simpler) code works just as well as it puts each stream directly into two separate strings that can be handled successfully within the tag. Not wishing to waste any more of yours or my time my inclination is to use this code as best I can.

Here is the code:

                def sOut = new StringBuilder()
                def sErr = new StringBuilder()
                def rC = attrs["runCmd"]
                def procCmd = rC.execute()
                procCmd.consumeProcessOutput(sOut, sErr)
                procCmd.waitFor()

                if (sErr) print (" RunCommand - Error: $sErr")
                if (sOut) print (" RunCommand1 - Output: $sOut")
       
Thanks for your help - this has all been very useful  it's introduced me to lots new grails ideas.


-Mike
Reply | Threaded
Open this post in threaded view
|

Re: capturing system command output

talldave
mike -

when you get into the world of command lines (and especially when it comes to trying to programmatically terminate processes) you will encounter differences depending on the OS/platform you're working with.  that said, from looking at your code snippet i do see a problem.  note the difference between these lines:

while ( (i=is?.read()) != -1 || (ie=isErr?.read()) != -1 ) {

while ( (i=isOut?.read())||(ie=isErr?.read()) != -1 ) {

in your code you're not checking that the output of "isOut" is != -1, so once there's no longer any output it will get stuck on that line.  if isOut?.read() returned 0 that would fail groovy truth and the program would continue, but -1 is true in the world of groovy truth and so it needs to be checked for explicitly.  i'm assuming this is just a simple typo on your part though.

try adding the != -1 and post the results.  i'm still curious to hear if it works for you.

good to know that the simpler code using consumeProcessOutput() works as well - although i assume with that approach it's necessary to wait until the command has completed before you can process any of the output.

hope that helps!

dave
Reply | Threaded
Open this post in threaded view
|

Re: capturing system command output

talldave
ok, so in looking a bit closer my analysis was slightly off, but i think the net result is the same.  double check your while loop logic.  basically:

def out = -1
def err = -1
println out != -1 || err != -1  // checking them individually evaluates to false, so while loop exits
println ( out || err ) != -1      // combining and then checking evaluates to true, so while loop never exits

sorry, just felt the need to correct myself.  :)
Reply | Threaded
Open this post in threaded view
|

Re: capturing system command output

mkavanagh
In reply to this post by Nathan Wells
Hi Talldave,

Your suggested correction to my code fixed the job.

The ffmpeg commands I am using in an  MS DOS environment only produce output on the error stream which is not ideal - but at least it works.

Thanks for your help.

-Mike