Virtual Interface Problems
2007-12-25 10:12:00
unacklnowledged BUG on the part of SunSoft. I can't believe that they
would consider round robin interfaces as a "feature" and enable it by
default.
I received many suggestions, most recommended I move virtual IP setup to
S99something in /etc/rc2.d. None of them, however, represented a complete
fix to the problem. Others suggested not actually turning up hme0 until
after the virtual interfaces had been configured.
I faced a couple of snags. 1) Disabling the interface group (round robin
IP allocation) seemed to _break_ all virtual interfaces, whether done
before or after virtual interface configuration. The problem seems to be
that the machine does _not_ respond to ARP requests for its virtual IP
addresses. This happens immediately after disabling group interfaces. The
other problem I faced was that arp and ndd are not available in the sigle
user startup phase of boot, so I had to move the entire virtual interface
loop out of rootuser.sh, which will make it difficult if I ever need to
mount /usr via NFS.
In order to get this to work at all, I had to (YMMV):
1) Move all of the interface configuration (virtual and otherwise) into
/etc/rc2.d/S99vif
2) In the beginning of S99vif, I had to disable group interfaces with
ndd -set /dev/ip ip_enable_group_ifs 0
3) I had to add static arp entries for all vif interfaces. This can be
done by putting
arp -s "$addr" <ETHADDR>
in the middle of the loop where it turns up the vif interfaces.
At this point, it uses the _last_ virtual interface configured as the
source address of all connections. In my S99 script, I tried to turn up
hme0 again, but for some reason that didn't take.
If I try to do another ifconfig on hme0 from the prompt after boot, it
will then use my primary IP for all outbound connections, but it will
BREAK virtual interfaces.
This is obviously a bug.
I've tried other hacks, such as making the vif metrics 1, but that did not
work.
If anyone else would like to pick up where I left off, we (me and several
other list members) would appreciate the help. This is a production
server, so I can't afford to be rebooting it constantly to test startup
scripts.
-Jason
----- Original Message ----
Solaris 2.6, Ultra 2170E
I'm trying to move my web server from my old SunOS box to the new solaris
machine. I'm working with about 150 virtual domains, each taking up a
virtual interface on hme0. I've made my /etc/hostname.hme0 and
/etc/hostname.hme0:1-hme0:xxx files and it's configuring the virtual
interfaces just fine. One of the things I noticed, was that when
establishing an outbound connection from the machine, it would randomly
pick one of the virtual interface IP addresses as the source address of
the connection. I managed to get around this by setting
ip_enable_group_ifs to 0 with ndd. At that point, it would use the last
interface I configured, so I'd just re ifconfig hme0 and everything worked
fine -- it would use the primary address.
This works find if I do it by hand at the shell after it's booted. BUT, if
I try to add these tasks (ndd and ifconfig hme0) to the end of startup
scripts like rootusr or inetsvc, my virtual interfaces will BREAK. At
this point, the only thing I can do to get them to work this way is add
static, published arp entries for each interface; however, if I perform
the ndd and the ifconfig from the shell prompt, everything works fine.
In fact the virtual interfaces work fine even if I don't do the ndd and
the ifconfig at the shell, its just that it picks random source addresses.
Is there something that I missed? I'm pretty baffled at this point. I
don't understand why something would work fine at the shell prompt but
completely break things in a startup script (I even tried putting it in
S80httpd in /etc/rc3.d to be sure other vital configs were done, with and
without explicit paths).
Any help would be appreciated. I've searched the archives, but there isn't
a summary that covers this problem exactly.
-Jason
Comments
Got something to say?
You must be logged in to post a comment.
