Thread: Quadranet downtown LA

I'm not super familiar with IPMI (as we use external hardware based Raritan KVM over IP units). The KVM over IP units we use support virtual media (IE: remote mounting an ISO, or a USB key, or what have you, to allow for BIOS updates, OS installs, etc.).

I any remotely current versions of IPMI have similar functionality ("virtual media" that is). Is it possible to mount a virtual media device, with read/write permissions, and then write to that media (IE: mounting a partition, path, or USB key say over IPMI)?

For the KVM over IP gear, transfer speeds are pretty slow (as the KVM dongle negotiates at 100mbps to the KVM switch, and then the virtual media only gets a chunk of that), but pretty slow data extraction is a heck of a lot better than none.

Originally Posted by iridium69

Normally SCP should do the trick but I get "no route to host" errors

try to post your server IP address.
If some members here can ping your server, there is a good chance that you can scp with the help of wht member

Originally Posted by sosys

try to post your server IP address.
If some members here can ping your server, there is a good chance that you can scp with the help of wht member

The server IP is 216.45.59.202
It isn't accessible/ping from outside but If someone is hosted in the LA datacenter and can try.. that would be great

Originally Posted by iridium69

The server IP is 216.45.59.202
It isn't accessible/ping from outside but If someone is hosted in the LA datacenter and can try.. that would be great

Code:

[root@serv2 csf]# traceroute 216.45.59.202
traceroute to 216.45.59.202 (216.45.59.202), 30 hops max, 60 byte packets
 1  66.63.177.113.static.quadranet.com (66.63.177.113)  1.667 ms  1.634 ms  1.618 ms
 2  * * *
 3  10.9.5.153 (10.9.5.153)  0.806 ms !N 10.9.4.37 (10.9.4.37)  0.544 ms !N 10.9.5.153 (10.9.5.153)  1.622 ms !N
[root@serv2 csf]# ping 216.45.59.202
PING 216.45.59.202 (216.45.59.202) 56(84) bytes of data.
From 10.9.5.153 icmp_seq=2 Destination Net Unreachable
From 10.9.5.153 icmp_seq=5 Destination Net Unreachable
From 10.9.5.153 icmp_seq=8 Destination Net Unreachable

Originally Posted by iridium69

The server IP is 216.45.59.202
It isn't accessible/ping from outside but If someone is hosted in the LA datacenter and can try.. that would be great

unfortunately it didnt work

Code:

root@backup:~ #	traceroute 216.45.59.202
traceroute to 216.45.59.202 (216.45.59.202), 64 hops max, 40 byte packets
 1  66.154.127.17.static.quadranet.com (66.154.127.17)  0.577 ms  1.219 ms  0.364 ms
 2  64.188.0.242 (64.188.0.242)  0.234 ms  0.272 ms  0.239 ms
 3  64.188.0.3.static.quadranet.com (64.188.0.3)  0.237 ms  0.277 ms  0.241 ms
 4  64.188.0.36.static.quadranet.com (64.188.0.36)  0.867 ms  0.819 ms  7.844 ms
 5  * * *
 6  *^C

@allanw @sosys Thanks a lot for trying this !
This confirms what I have been suspecting , ie. that the box is either not physically connected to the local network or that switching/routing isn't configured correctly.
What I don't understand is in that case how come I am able to connect through IPMI .. if such routes are possible it cannot be that difficult for them to provide minimum connectivity?

The server is configured to reply to a ICMP (ping) - Yes? If not, then give us a running service / port number to telnet to, and see if it responds.

Originally Posted by iridium69

@allanw @sosys Thanks a lot for trying this !
This confirms what I have been suspecting , ie. that the box is either not physically connected to the local network or that switching/routing isn't configured correctly.
What I don't understand is in that case how come I am able to connect through IPMI .. if such routes are possible it cannot be that difficult for them to provide minimum connectivity?

are you being given public IP address to access IPMI ? or are you being given KVM lantronix access ?
for my case, i have to reconfigure eth0 with public IP address.
Previously, eth0 was the private network, and eth1 was the public network.

Originally Posted by sosys

are you being given public IP address to access IPMI ? or are you being given KVM lantronix access ?
for my case, i have to reconfigure eth0 with public IP address.
Previously, eth0 was the private network, and eth1 was the public network.

To access IPMI they gave me a specific public IP, this IP is new and not related to the public IP ranges I had on the server.
eth0 was reconfigured with my primary public IP (216....)

I've been reading this thread. This looks really bad. We also had some customers leasing QN IPS, they were all of a sudden sold out and derouted/assigned to another AS. We have been escalating this issue for weeks now. Nobody really cares.

Originally Posted by iridium69

To access IPMI they gave me a specific public IP, this IP is new and not related to the public IP ranges I had on the server.
eth0 was reconfigured with my primary public IP (216....)

is this the same IP prior to all of these incidents ?
try to use the original IP address that you were assigned when you purchased the server.
Or try other IP subnet that you were assigned in the past.

I presume your gateway is 216.45.59.201, this is not pingable from outside
I can ping this gateway IP from QN server,but not from my computer

Code:

root@backup:~ #	ping 216.45.59.201
PING 216.45.59.201 (216.45.59.201): 56 data bytes
64 bytes from 216.45.59.201: icmp_seq=0 ttl=61 time=1.615 ms
64 bytes from 216.45.59.201: icmp_seq=1 ttl=61 time=1.372 ms
64 bytes from 216.45.59.201: icmp_seq=2 ttl=61 time=3.970 ms
64 bytes from 216.45.59.201: icmp_seq=3 ttl=61 time=2.518 ms

Did you try to email Jon ? Probably he can help to direct you to his network engineer

Originally Posted by sosys

is this the same IP prior to all of these incidents ?
try to use the original IP address that you were assigned when you purchased the server.
Or try other IP subnet that you were assigned in the past.

I presume your gateway is 216.45.59.201, this is not pingable from outside
I can ping this gateway IP from QN server,but not from my computer

Code:

root@backup:~ #	ping 216.45.59.201
PING 216.45.59.201 (216.45.59.201): 56 data bytes
64 bytes from 216.45.59.201: icmp_seq=0 ttl=61 time=1.615 ms
64 bytes from 216.45.59.201: icmp_seq=1 ttl=61 time=1.372 ms
64 bytes from 216.45.59.201: icmp_seq=2 ttl=61 time=3.970 ms
64 bytes from 216.45.59.201: icmp_seq=3 ttl=61 time=2.518 ms

Did you try to email Jon ? Probably he can help to direct you to his network engineer

On 2nd thought, since i can reach your gateway, did you configure your subnet correctly ?
maybe give your port as rosswht mentioned, hopefully there is a good chance some of WHT members can reach your server

Originally Posted by sosys

On 2nd thought, since i can reach your gateway, did you configure your subnet correctly ?
maybe give your port as rosswht mentioned, hopefully there is a good chance some of WHT members can reach your server

Thanks again for your help looking into this ! The fact you can ping the gateway is good news..
All standard ports should be running (80,443,22...); configuration was done by the QN tech to revert public on eth0, the IP is the original one...

It seems there is a problem starting network services on the server. I get the following error :

Failed to start LSB: Bring up/down networking

So probably the tech misconfigured something, but everything looks normal (but I'm not a networking specialist, this server was supposedly "fully managed" )

Originally Posted by UnfinishedSentenc

Over a week now and they still don't have the VPN working. No new updates on the status page for a few days now.

Yeah something is really strange with the VPN situation. They won't say a word about resolving it. And it's also strange that it went down at all because I don't think these IP's should have been affected. At least one of the two endpoints should have continued working, and if it was IP related that would be an easy fix that they surely would have done by now. I wonder if they did something really stupid like repurposed the machines... or sold them off or never acquired them in the recent dealings.

Originally Posted by Milychlapik

It is better to cancel with a partial refund and move everything off their network. For me it is clear their services will never improve.

Yeah, it would be impossible to trust them enough to put critical, non-redundant services there, overseen by the team that let this ridiculousness happen. This greatly reduces the value so I am terminating. Such a spectacular implosion for a once great data center.

Originally Posted by iridium69

Thanks again for your help looking into this ! The fact you can ping the gateway is good news..
All standard ports should be running (80,443,22...); configuration was done by the QN tech to revert public on eth0, the IP is the original one...

It seems there is a problem starting network services on the server. I get the following error :

Failed to start LSB: Bring up/down networking

So probably the tech misconfigured something, but everything looks normal (but I'm not a networking specialist, this server was supposedly "fully managed" )

this means some configuration could be missing (like DEFROUTE).
Do you want to paste your eth1 config to eth0 ? just see if there is network property that should be there.
I guess once your network is up, you should be able to connect to the internet.

as you are now having eth0 which also sharing with IPMI, i wonder why the IPMI works (for sure QN assigned a public IP) but when start the network it failed.

Try a manual ifdown eth0; ifup eth0

Otherwise, you can post your ifcf-eth0, ifcf-eth1, and /etc/sysconfig/network in here.
There are tons of experts watching this thread and willing to help you to get UP

Originally Posted by sosys

this means some configuration could be missing (like DEFROUTE).
Do you want to paste your eth1 config to eth0 ? just see if there is network property that should be there.
I guess once your network is up, you should be able to connect to the internet.

as you are now having eth0 which also sharing with IPMI, i wonder why the IPMI works (for sure QN assigned a public IP) but when start the network it failed.

Try a manual ifdown eth0; ifup eth0

Otherwise, you can post your ifcf-eth0, ifcf-eth1, and /etc/sysconfig/network in here.
There are tons of experts watching this thread and willing to help you to get UP

My ports are eno1 and eno2 ... eno2 config has been blanked out with # at each line
eno1 looks like this

# Public 216.45.59.200/29
DEVICE=eno1
BOOTPROTO=static
HWADDR=AC:1F:6B:7B:E6:46
ONBOOT=yes
NM_CONTROLLED=no
NOZEROCONF=yes
IPADDR= 216.45.59.202
NETMASK=255.255.255.248
GATEWAY= 216.45.59.201
DNS1=8.8.8.8

The port HWADDR is correct . other settings look like the old ones on eno2

ethtool eno1 shows the port is active link detected full-duplex etc (not the case with eno2 which QN disconnected)

/etc/sysconfig/network is a blank file

Originally Posted by sosys

as you are now having eth0 which also sharing with IPMI, i wonder why the IPMI works (for sure QN assigned a public IP) but when start the network it failed.

and yes, why IPMI works is beyond my understanding. They were very quick to supply me with IPMI credentials and a specific IP to login.

Maybe this IPMI connection goes through some sort of separate module they plugged in the server ?

Otherwise shouldn't I be able to hack that IP to reach the server somehow?

Originally Posted by iridium69

My ports are eno1 and eno2 ... eno2 config has been blanked out with # at each line
eno1 looks like this

# Public 216.45.59.200/29
DEVICE=eno1
BOOTPROTO=static
HWADDR=AC:1F:6B:7B:E6:46
ONBOOT=yes
NM_CONTROLLED=no
NOZEROCONF=yes
IPADDR= 216.45.59.202
NETMASK=255.255.255.248
GATEWAY= 216.45.59.201
DNS1=8.8.8.8

The port HWADDR is correct . other settings look like the old ones on eno2

ethtool eno1 shows the port is active link detected full-duplex etc (not the case with eno2 which QN disconnected)

/etc/sysconfig/network is a blank file

is it due to the extra space in your IPADDR and GATEWAY ?

try this:

Code:

ip addr add 216.45.59.202/29 dev eno1
ip route add default via 216.45.59.201

If still does not work, share this output:

Code:

ip a show eno1
ip route
systemctl status network
journalctl -xe | tail -50

Originally Posted by sosys

is it due to the extra space in your IPADDR and GATEWAY ?

no just a typo in pasting here

try this:

Code:

ip addr add 216.45.59.202/29 dev eno1
ip route add default via 216.45.59.201

added both and did some cleanups but not working

Code:

[root@172-16-4-195 network-scripts]# ip a show eno12: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mg state UP group default qlen 1000
    link/ether ac:1f:6b:7b:e6:46 brd ff:ff:ff:ff:ff:ff
    inet 216.45.59.202/29 brd 216.45.59.207 scope global eno1
        valid_lft forever preferred_lft forever
    inet 216.45.59.202/32 scope global eno1
        valid_lft forever preferred_lft forever
    inet 216.45.59.203/29 brd 216.45.59.207 scope global secondary eno1:1
        valid_lft forever preferred_lft forever
    inet 216.45.59.204/29 brd 216.45.59.207 scope global secondary eno1:2
        valid_lft forever preferred_lft forever
    inet 216.45.59.205/29 brd 216.45.59.207 scope global secondary eno1:3
        valid_lft forever preferred_lft forever
    inet 216.45.59.206/29 brd 216.45.59.207 scope global secondary eno1:4
        valid_lft forever preferred_lft forever
    inet6 fe80::ae1f:6bff:fe7b:e646/64 scope link 
        valid_lft forever preferred_lft forever
[root@172-16-4-195 network-scripts]#

Code:

[root@172-16-4-195 network-scripts]# ip route
default via 216.45.59.201 dev eno1
192.161.62.0/28 dev eno2 proto kernel scope link src 192.161.62.1
192.161.62.1 dev eno2 scope link src 192.161.62.1
192.161.62.2 dev eno2 scope link src 192.161.62.2
192.161.62.3 dev eno2 scope link src 192.161.62.3
192.161.62.4 dev eno2 scope link src 192.161.62.4
192.161.62.5 dev eno2 scope link src 192.161.62.5
192.161.62.6 dev eno2 scope link src 192.161.62.6
192.161.62.7 dev eno2 scope link src 192.161.62.7
192.161.62.8 dev eno2 scope link src 192.161.62.8
192.161.62.9 dev eno2 scope link src 192.161.62.9
192.161.62.10 dev eno2 scope link src 192.161.62.10
192.161.62.11 dev eno2 scope link src 192.161.62.11
192.161.62.12 dev eno2 scope link src 192.161.62.12
192.161.62.13 dev eno2 scope link src 192.161.62.13
192.161.62.14 dev eno2 scope link src 192.161.62.14
216.45.59.200/29 dev eno1 proto kernel scope link src 216.45.59.202
[root@172-16-4-195 network-scripts]#

Code:

[root@172-16-4-195 network-scripts]# systemctl status network
● network.service - LSB: Bring up/down networking
   Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled)
   Active: failed (Result: exit-code) since Sat 28:100:29.2038:37:46 PDT; 16min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 5425 ExecStart=/etc/rc.d/init.d/network start (code=exited, status=1/FAILURE)


Sep 29 08:37:46 172-16-4-195.cprapid.com network[5425]: RTNETLINK answers: File exists
Sep 29 08:37:46 172-16-4-195.cprapid.com network[5425]: RTNETLINK answers: File exists
Sep 29 08:37:46 172-16-4-195.cprapid.com network[5425]: RTNETLINK answers: File exists
Sep 29 08:37:46 172-16-4-195.cprapid.com network[5425]: RTNETLINK answers: File exists
Sep 29 08:37:46 172-16-4-195.cprapid.com network[5425]: RTNETLINK answers: File exists
Sep 29 08:37:46 172-16-4-195.cprapid.com systemd[1]: network.service: control process exited, code=exited status=1
Sep 29 08:37:46 172-16-4-195.cprapid.com systemd[1]: Failed to start LSB: Bring up/down networking.
Sep 29 08:37:46 172-16-4-195.cprapid.com systemd[1]: Unit network.service entered failed state.
Sep 29 08:37:46 172-16-4-195.cprapid.com systemd[1]: network.service failed.
[root@172-16-4-195 network-scripts]#

im not sure why you have this entry:

Code:

    inet 216.45.59.202/32 scope global eno1
        valid_lft forever preferred_lft forever

Maybe restart again the eno1:

Code:

 ifdown eno1; ifup eno1

Then add again the IP address via ip addr and ip route add

@sosys It's not working unfortunately. Stil many thanks for the trouble trying to help me out!
I think you are very close with the specific configs you told me to look at, but there is something beyond our control that probably only Quadranet would be able to sort out, if they finally got to it.
Will see tomorrow if they recover from the weekend and will keep people here posted.
In the meantime, wishing you a good evening/night !

Still down i dont know why support not replying anymore

Originally Posted by iridium69

and yes, why IPMI works is beyond my understanding. They were very quick to supply me with IPMI credentials and a specific IP to login.

Maybe this IPMI connection goes through some sort of separate module they plugged in the server ?

Otherwise shouldn't I be able to hack that IP to reach the server somehow?

IPMI runs on a standalone micro controller.
If IPMI shares the port with server, the nic is probably configured with two mac address.

Just cancelled all my servers with them. These clowns are ridiculous.

Originally Posted by myocytebd

IPMI runs on a standalone micro controller.
If IPMI shares the port with server, the nic is probably configured with two mac address.

If they can supply a public IP to run IPMI (and it is working), it shouldn't be much trouble to supply and configure an ip to access the server directly?
Isn't it pretty much a question of taking 5-10 minutes to configure this or am I missing something?