Too many open files

bridged with qdn.public.devtools
Post Reply
Johannes

Too many open files

Post by Johannes » Tue Aug 26, 2003 11:28 am

My program has encountered "Too many open files" error when it spawns
processes. So I modify /boot/build/qnxbasedma.build file by adding the
option -F 1000 to procnto, and invoke mkifs qnxbasedma.build /.boot. And
reboot the system. But the problem remains. Can anyone tell me what I have
missed here? Am I modifying the right build file and command? Thanks.

Jens H Jorgensen

Re: Too many open files

Post by Jens H Jorgensen » Tue Aug 26, 2003 11:55 am

Could you have a file descriptor leak in your program?

We had the same problem and it was due to a fd leak in our software. Use
something like spin to monitor the number of file descriptors.

--
Jens


"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:biffh4$1ao$1@inn.qnx.com...
My program has encountered "Too many open files" error when it spawns
processes. So I modify /boot/build/qnxbasedma.build file by adding the
option -F 1000 to procnto, and invoke mkifs qnxbasedma.build /.boot. And
reboot the system. But the problem remains. Can anyone tell me what I have
missed here? Am I modifying the right build file and command? Thanks.

Johannes

Re: Too many open files

Post by Johannes » Thu Aug 28, 2003 1:08 pm

Sorry, I was wrong. The error was not due to opening file, but creating
socket. What is the maximum socket that a process can create? How to set it
to the maximum? spin shows 878 FDs on both client and server processes and
the total FDs is over 2 thousands. sysconf (_SC_OPEN_MAX) is 10000 because I
have added -F10000 in procnto.

Where can I find this and other system limitations on the document?

Thanks

"Jens H Jorgensen" <jhj@remove-nospam-videk.com> wrote in message
news:bifh5c$227$1@inn.qnx.com...
Could you have a file descriptor leak in your program?

We had the same problem and it was due to a fd leak in our software. Use
something like spin to monitor the number of file descriptors.

--
Jens


"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:biffh4$1ao$1@inn.qnx.com...
My program has encountered "Too many open files" error when it spawns
processes. So I modify /boot/build/qnxbasedma.build file by adding the
option -F 1000 to procnto, and invoke mkifs qnxbasedma.build /.boot. And
reboot the system. But the problem remains. Can anyone tell me what I
have
missed here? Am I modifying the right build file and command? Thanks.



Sean Boudreau

Re: Too many open files

Post by Sean Boudreau » Thu Aug 28, 2003 2:48 pm

This should be fixed in 6.2.1B. The stack was enforcing
an artifically low limit.

-seanb

Johannes <Jsukamtoh@infolink.co.id> wrote:
Sorry, I was wrong. The error was not due to opening file, but creating
socket. What is the maximum socket that a process can create? How to set it
to the maximum? spin shows 878 FDs on both client and server processes and
the total FDs is over 2 thousands. sysconf (_SC_OPEN_MAX) is 10000 because I
have added -F10000 in procnto.

Where can I find this and other system limitations on the document?

Thanks

"Jens H Jorgensen" <jhj@remove-nospam-videk.com> wrote in message
news:bifh5c$227$1@inn.qnx.com...
Could you have a file descriptor leak in your program?

We had the same problem and it was due to a fd leak in our software. Use
something like spin to monitor the number of file descriptors.

--
Jens


"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:biffh4$1ao$1@inn.qnx.com...
My program has encountered "Too many open files" error when it spawns
processes. So I modify /boot/build/qnxbasedma.build file by adding the
option -F 1000 to procnto, and invoke mkifs qnxbasedma.build /.boot. And
reboot the system. But the problem remains. Can anyone tell me what I
have
missed here? Am I modifying the right build file and command? Thanks.



Johannes

Re: Too many open files

Post by Johannes » Mon Sep 01, 2003 1:33 am

Hi, Sean, I am using PE 6.2.1B version. And I have double check that with my
QNX installed software.

"Sean Boudreau" <seanb@node25.ott.qnx.com> wrote in message
news:bil4n1$225$1@nntp.qnx.com...
This should be fixed in 6.2.1B. The stack was enforcing
an artifically low limit.

-seanb

Johannes <Jsukamtoh@infolink.co.id> wrote:
Sorry, I was wrong. The error was not due to opening file, but creating
socket. What is the maximum socket that a process can create? How to set
it
to the maximum? spin shows 878 FDs on both client and server processes
and
the total FDs is over 2 thousands. sysconf (_SC_OPEN_MAX) is 10000
because I
have added -F10000 in procnto.

Where can I find this and other system limitations on the document?

Thanks

"Jens H Jorgensen" <jhj@remove-nospam-videk.com> wrote in message
news:bifh5c$227$1@inn.qnx.com...
Could you have a file descriptor leak in your program?

We had the same problem and it was due to a fd leak in our software.
Use
something like spin to monitor the number of file descriptors.

--
Jens


"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:biffh4$1ao$1@inn.qnx.com...
My program has encountered "Too many open files" error when it spawns
processes. So I modify /boot/build/qnxbasedma.build file by adding
the
option -F 1000 to procnto, and invoke mkifs qnxbasedma.build /.boot.
And
reboot the system. But the problem remains. Can anyone tell me what I
have
missed here? Am I modifying the right build file and command? Thanks.





Johannes

Re: Too many open files

Post by Johannes » Mon Sep 01, 2003 12:25 pm

Sorry, another mistake here. I forgot to increase -F in my boot image. It
works after I have done that with 6.2.1B. Thanks.


"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:biu6uo$t17$1@inn.qnx.com...
Hi, Sean, I am using PE 6.2.1B version. And I have double check that with
my
QNX installed software.

"Sean Boudreau" <seanb@node25.ott.qnx.com> wrote in message
news:bil4n1$225$1@nntp.qnx.com...

This should be fixed in 6.2.1B. The stack was enforcing
an artifically low limit.

-seanb

Johannes <Jsukamtoh@infolink.co.id> wrote:
Sorry, I was wrong. The error was not due to opening file, but
creating
socket. What is the maximum socket that a process can create? How to
set
it
to the maximum? spin shows 878 FDs on both client and server processes
and
the total FDs is over 2 thousands. sysconf (_SC_OPEN_MAX) is 10000
because I
have added -F10000 in procnto.

Where can I find this and other system limitations on the document?

Thanks

"Jens H Jorgensen" <jhj@remove-nospam-videk.com> wrote in message
news:bifh5c$227$1@inn.qnx.com...
Could you have a file descriptor leak in your program?

We had the same problem and it was due to a fd leak in our software.
Use
something like spin to monitor the number of file descriptors.

--
Jens


"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:biffh4$1ao$1@inn.qnx.com...
My program has encountered "Too many open files" error when it
spawns
processes. So I modify /boot/build/qnxbasedma.build file by adding
the
option -F 1000 to procnto, and invoke mkifs qnxbasedma.build
/.boot.
And
reboot the system. But the problem remains. Can anyone tell me what
I
have
missed here? Am I modifying the right build file and command?
Thanks.







Johannes

Re: Too many open files

Post by Johannes » Thu Sep 04, 2003 11:23 am

Now the problem turns to spawn function. I have set -F20000 to procnto, but
I still get EMFILE when I call spawn function. spin shows a total of 4079
FDs. Any ideas?

"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:bivd4m$q9q$1@inn.qnx.com...
Sorry, another mistake here. I forgot to increase -F in my boot image. It
works after I have done that with 6.2.1B. Thanks.


"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:biu6uo$t17$1@inn.qnx.com...
Hi, Sean, I am using PE 6.2.1B version. And I have double check that
with
my
QNX installed software.

"Sean Boudreau" <seanb@node25.ott.qnx.com> wrote in message
news:bil4n1$225$1@nntp.qnx.com...

This should be fixed in 6.2.1B. The stack was enforcing
an artifically low limit.

-seanb

Johannes <Jsukamtoh@infolink.co.id> wrote:
Sorry, I was wrong. The error was not due to opening file, but
creating
socket. What is the maximum socket that a process can create? How to
set
it
to the maximum? spin shows 878 FDs on both client and server
processes
and
the total FDs is over 2 thousands. sysconf (_SC_OPEN_MAX) is 10000
because I
have added -F10000 in procnto.

Where can I find this and other system limitations on the document?

Thanks

"Jens H Jorgensen" <jhj@remove-nospam-videk.com> wrote in message
news:bifh5c$227$1@inn.qnx.com...
Could you have a file descriptor leak in your program?

We had the same problem and it was due to a fd leak in our
software.
Use
something like spin to monitor the number of file descriptors.

--
Jens


"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:biffh4$1ao$1@inn.qnx.com...
My program has encountered "Too many open files" error when it
spawns
processes. So I modify /boot/build/qnxbasedma.build file by
adding
the
option -F 1000 to procnto, and invoke mkifs qnxbasedma.build
/.boot.
And
reboot the system. But the problem remains. Can anyone tell me
what
I
have
missed here? Am I modifying the right build file and command?
Thanks.









Jens H Jorgensen

Re: Too many open files

Post by Jens H Jorgensen » Thu Sep 04, 2003 3:14 pm

This is a side question - but once again why do you that many open sockets?

--
Jens
"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:bj76k9$jp9$1@inn.qnx.com...
Now the problem turns to spawn function. I have set -F20000 to procnto,
but
I still get EMFILE when I call spawn function. spin shows a total of 4079
FDs. Any ideas?

"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:bivd4m$q9q$1@inn.qnx.com...
Sorry, another mistake here. I forgot to increase -F in my boot image.
It
works after I have done that with 6.2.1B. Thanks.


"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:biu6uo$t17$1@inn.qnx.com...
Hi, Sean, I am using PE 6.2.1B version. And I have double check that
with
my
QNX installed software.

"Sean Boudreau" <seanb@node25.ott.qnx.com> wrote in message
news:bil4n1$225$1@nntp.qnx.com...

This should be fixed in 6.2.1B. The stack was enforcing
an artifically low limit.

-seanb

Johannes <Jsukamtoh@infolink.co.id> wrote:
Sorry, I was wrong. The error was not due to opening file, but
creating
socket. What is the maximum socket that a process can create? How
to
set
it
to the maximum? spin shows 878 FDs on both client and server
processes
and
the total FDs is over 2 thousands. sysconf (_SC_OPEN_MAX) is 10000
because I
have added -F10000 in procnto.

Where can I find this and other system limitations on the
document?

Thanks

"Jens H Jorgensen" <jhj@remove-nospam-videk.com> wrote in message
news:bifh5c$227$1@inn.qnx.com...
Could you have a file descriptor leak in your program?

We had the same problem and it was due to a fd leak in our
software.
Use
something like spin to monitor the number of file descriptors.

--
Jens


"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:biffh4$1ao$1@inn.qnx.com...
My program has encountered "Too many open files" error when it
spawns
processes. So I modify /boot/build/qnxbasedma.build file by
adding
the
option -F 1000 to procnto, and invoke mkifs qnxbasedma.build
/.boot.
And
reboot the system. But the problem remains. Can anyone tell me
what
I
have
missed here? Am I modifying the right build file and command?
Thanks.











Johannes

Re: Too many open files

Post by Johannes » Thu Sep 04, 2003 4:52 pm

Because my application is interfacing with hundreds of devices which can
communicate on socket only, and also because I am using UDP for IPC instead
of QNX messaging for the sake of running the application on other O/S.

"Jens H Jorgensen" <jhj@remove-nospam-videk.com> wrote in message
news:bj7k58$t53$1@inn.qnx.com...
This is a side question - but once again why do you that many open
sockets?

--
Jens
"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:bj76k9$jp9$1@inn.qnx.com...
Now the problem turns to spawn function. I have set -F20000 to procnto,
but
I still get EMFILE when I call spawn function. spin shows a total of
4079
FDs. Any ideas?

"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:bivd4m$q9q$1@inn.qnx.com...
Sorry, another mistake here. I forgot to increase -F in my boot image.
It
works after I have done that with 6.2.1B. Thanks.


"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:biu6uo$t17$1@inn.qnx.com...
Hi, Sean, I am using PE 6.2.1B version. And I have double check that
with
my
QNX installed software.

"Sean Boudreau" <seanb@node25.ott.qnx.com> wrote in message
news:bil4n1$225$1@nntp.qnx.com...

This should be fixed in 6.2.1B. The stack was enforcing
an artifically low limit.

-seanb

Johannes <Jsukamtoh@infolink.co.id> wrote:
Sorry, I was wrong. The error was not due to opening file, but
creating
socket. What is the maximum socket that a process can create?
How
to
set
it
to the maximum? spin shows 878 FDs on both client and server
processes
and
the total FDs is over 2 thousands. sysconf (_SC_OPEN_MAX) is
10000
because I
have added -F10000 in procnto.

Where can I find this and other system limitations on the
document?

Thanks

"Jens H Jorgensen" <jhj@remove-nospam-videk.com> wrote in
message
news:bifh5c$227$1@inn.qnx.com...
Could you have a file descriptor leak in your program?

We had the same problem and it was due to a fd leak in our
software.
Use
something like spin to monitor the number of file descriptors.

--
Jens


"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:biffh4$1ao$1@inn.qnx.com...
My program has encountered "Too many open files" error when
it
spawns
processes. So I modify /boot/build/qnxbasedma.build file by
adding
the
option -F 1000 to procnto, and invoke mkifs qnxbasedma.build
/.boot.
And
reboot the system. But the problem remains. Can anyone tell
me
what
I
have
missed here? Am I modifying the right build file and command?
Thanks.













Johannes

Re: Too many open files

Post by Johannes » Fri Sep 05, 2003 2:14 am

Hi Jen, any idea what is the maximum number of socket that Neutrino
supports? This is the question I have in mind, including other system
limitations that nobody from QNX cares to answer it

"Jens H Jorgensen" <jhj@remove-nospam-videk.com> wrote in message
news:bj7k58$t53$1@inn.qnx.com...
This is a side question - but once again why do you that many open
sockets?

--
Jens
"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:bj76k9$jp9$1@inn.qnx.com...
Now the problem turns to spawn function. I have set -F20000 to procnto,
but
I still get EMFILE when I call spawn function. spin shows a total of
4079
FDs. Any ideas?

"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:bivd4m$q9q$1@inn.qnx.com...
Sorry, another mistake here. I forgot to increase -F in my boot image.
It
works after I have done that with 6.2.1B. Thanks.


"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:biu6uo$t17$1@inn.qnx.com...
Hi, Sean, I am using PE 6.2.1B version. And I have double check that
with
my
QNX installed software.

"Sean Boudreau" <seanb@node25.ott.qnx.com> wrote in message
news:bil4n1$225$1@nntp.qnx.com...

This should be fixed in 6.2.1B. The stack was enforcing
an artifically low limit.

-seanb

Johannes <Jsukamtoh@infolink.co.id> wrote:
Sorry, I was wrong. The error was not due to opening file, but
creating
socket. What is the maximum socket that a process can create?
How
to
set
it
to the maximum? spin shows 878 FDs on both client and server
processes
and
the total FDs is over 2 thousands. sysconf (_SC_OPEN_MAX) is
10000
because I
have added -F10000 in procnto.

Where can I find this and other system limitations on the
document?

Thanks

"Jens H Jorgensen" <jhj@remove-nospam-videk.com> wrote in
message
news:bifh5c$227$1@inn.qnx.com...
Could you have a file descriptor leak in your program?

We had the same problem and it was due to a fd leak in our
software.
Use
something like spin to monitor the number of file descriptors.

--
Jens


"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:biffh4$1ao$1@inn.qnx.com...
My program has encountered "Too many open files" error when
it
spawns
processes. So I modify /boot/build/qnxbasedma.build file by
adding
the
option -F 1000 to procnto, and invoke mkifs qnxbasedma.build
/.boot.
And
reboot the system. But the problem remains. Can anyone tell
me
what
I
have
missed here? Am I modifying the right build file and command?
Thanks.













Jens H Jorgensen

Re: Too many open files

Post by Jens H Jorgensen » Fri Sep 05, 2003 12:39 pm

Hello Johannes

No unfortunatly I don't.

Do you have an active support agreement with QSSL?

If that is the case send an email to support@"remove-nospam"qnx.com

--
Jens

"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:bj8qr5$poj$1@inn.qnx.com...
Hi Jen, any idea what is the maximum number of socket that Neutrino
supports? This is the question I have in mind, including other system
limitations that nobody from QNX cares to answer it

"Jens H Jorgensen" <jhj@remove-nospam-videk.com> wrote in message
news:bj7k58$t53$1@inn.qnx.com...
This is a side question - but once again why do you that many open
sockets?

--
Jens
"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:bj76k9$jp9$1@inn.qnx.com...
Now the problem turns to spawn function. I have set -F20000 to
procnto,
but
I still get EMFILE when I call spawn function. spin shows a total of
4079
FDs. Any ideas?

"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:bivd4m$q9q$1@inn.qnx.com...
Sorry, another mistake here. I forgot to increase -F in my boot
image.
It
works after I have done that with 6.2.1B. Thanks.


"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:biu6uo$t17$1@inn.qnx.com...
Hi, Sean, I am using PE 6.2.1B version. And I have double check
that
with
my
QNX installed software.

"Sean Boudreau" <seanb@node25.ott.qnx.com> wrote in message
news:bil4n1$225$1@nntp.qnx.com...

This should be fixed in 6.2.1B. The stack was enforcing
an artifically low limit.

-seanb

Johannes <Jsukamtoh@infolink.co.id> wrote:
Sorry, I was wrong. The error was not due to opening file, but
creating
socket. What is the maximum socket that a process can create?
How
to
set
it
to the maximum? spin shows 878 FDs on both client and server
processes
and
the total FDs is over 2 thousands. sysconf (_SC_OPEN_MAX) is
10000
because I
have added -F10000 in procnto.

Where can I find this and other system limitations on the
document?

Thanks

"Jens H Jorgensen" <jhj@remove-nospam-videk.com> wrote in
message
news:bifh5c$227$1@inn.qnx.com...
Could you have a file descriptor leak in your program?

We had the same problem and it was due to a fd leak in our
software.
Use
something like spin to monitor the number of file
descriptors.

--
Jens


"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:biffh4$1ao$1@inn.qnx.com...
My program has encountered "Too many open files" error when
it
spawns
processes. So I modify /boot/build/qnxbasedma.build file by
adding
the
option -F 1000 to procnto, and invoke mkifs
qnxbasedma.build
/.boot.
And
reboot the system. But the problem remains. Can anyone tell
me
what
I
have
missed here? Am I modifying the right build file and
command?
Thanks.















Sean Boudreau

Re: Too many open files

Post by Sean Boudreau » Fri Sep 05, 2003 1:14 pm

6.2.1B is only limited by available memory. Any per process limits
are enforced by procnto. Does the process you're spawning need to
inherit all these sockets? If not, set fcntl(sock, F_SETFD, FD_CLOEXEC).

Run multiple instances of the following if you want to test system wide
socket limits:

-seanb



#include <sys/socket.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>

int
main(void)
{
int i;


for (i = 0; ; i++) {
if (socket(AF_INET, SOCK_STREAM, 0) == -1) {
fprintf(stderr, "socket # %d: %s\n", i, strerror(errno));
break;
}
}

pause();

return 0;
}

Johannes <Jsukamtoh@infolink.co.id> wrote:
Hi Jen, any idea what is the maximum number of socket that Neutrino
supports? This is the question I have in mind, including other system
limitations that nobody from QNX cares to answer it

"Jens H Jorgensen" <jhj@remove-nospam-videk.com> wrote in message
news:bj7k58$t53$1@inn.qnx.com...
This is a side question - but once again why do you that many open
sockets?

--
Jens
"Johannes" <Jsukamtoh@infolink.co.id> wrote in message
news:bj76k9$jp9$1@inn.qnx.com...
Now the problem turns to spawn function. I have set -F20000 to procnto,
but
I still get EMFILE when I call spawn function. spin shows a total of
4079
FDs. Any ideas?

Post Reply

Return to “qdn.public.devtools”