DHCP Lease Times for Static IP Addresses

It appears that the max lease time in the XG is 30 days.

I don't think any of the settings you changed on the new APs should make a difference. Are there any other security settings on the APs that would keep a device from changing its IP or block certain packets? Do the APs have an internal DHCP server that's configured strangely and not completely offline?

You can try capturing packets on the XG from one of the devices and look for the difference between what it sends on a power cycle and what it sends when the lease expires. That might provide a clue.

Can you tell when the light goes offline? Is it at half of the lease length, 90% of the lease length, or 100% of the lease length? I just know what I read online about this process, but according to what I read, at 50% of the lease time the device reaches out to the DHCP server from whence it got its lease. If it goes to 90%, it will generate a broadcast -- as it does when it powers up -- asking for any DHCP server to respond. So my guess is that something is going wrong with the renewal (aT 50%) that's locking up the lights (which don't see or don't understand the reply) and they don't go on to do the broadcast request. So you power cycle them and they do.

My guess would be the AP is somehow not passing the XG's renewal message through, maybe due to a security setting in the AP to stop unsolicited "renewals" from changing IP addresses.

I didn't think it had anything to do with AP settings, but low hanging fruit....

There really aren't any security settings other than the authentication method. There is no DHCP on the AP's either, so no dice There.

I'm not sure how to capture packets - if you could provide instruction or point me to a document or something that would be great. 

It appears as though it is employing the 50% rule.  Maybe the XG is blocking broadcasts?  I see in the logs all the time denied traffic that is going to xx.xx.xx.255 - is that a broadcast?  How do I create a rule to allow that?  I think you may be on to something with that thought. As soon as I'm back in my office I'm going to chase that line of reasoning. 

Yes .255 is broadcast. But it's backwards from your problem, which is that DHCP works when the device initially broadcasts its request, but it doesn't work when it goes directly to the DHCP server itself to renew the lease. It could be that the device is doing things in a slightly strange way that the XG doesn't quite like, I guess, too.

In the GUI, under Diagnostics, there is a Packet Capture. You specify what Packets to capture with a BPF-language filter, In your case, I think you need one like ether host aa:bb:cc:dd:ee:ff where aa:bb:cc:dd:ee:ff is the MAC address of one of your problematic devices. Let it run for the full cycle -- there shouldn't be much traffic, and you might need to use the Refresh button to update the display so you see it. When you've gone through the entire cycle from before power-up to after it's offline, you can stop the capture and look at what packets you got, and hence get an idea of what the device was asking and what it should have received back.

Is it also possible to put the Netgear router-as-AP back into the mix just to test this and eliminate it as an AP-related issue?

OK, so I got the packet capture going and there is a ton of communication for such a simple device.  After just a few minutes, I have 8 pages of information.  The only thing I found that was interesting was this:

Do you have any idea what this means?  I will let it keep running until it loses connection and start analyzing the information.

I can put the other AP back into the mix, but I have it performing other duties and it will take a while to reset it and get it reconfigured for this purpose.

Good catch. Port 67 is a BOOTP port and also DHCP so perhaps that is the issue. If the 10.30.20.35 is the IP address of your device, it does look like it's attempting to do something DHCP-related directly to the server. (I assume that 20.30.20.1 is the firewall's IP on that subnet, and hence is the DHCP server.)

So it looks like there was a violation that I'm guessing is the reason things are failing as they are. Not sure what the Local Access Control List is, exactly, but it's evidently not allowing some devices (or a subnet) to talk to port 67. And that would fit your symptoms: power-on is a broadcast which doesn't violate the ACL, but renewal is a direct connection which does.

Hi,

have you created list of devices that use the ACL control access on your network?

I have one devcvie  on my network (IoT) that has a static address that is renews every 5 minutes when it connects and performs updates. I do see a lot of port 66/67 errors as well as successful renews. I did create  thread on the subject but never received any what I would call meaningful answers, so I left the device as is because it is working correctly.

Ian

You are correct on your assumptions.  10.30.20.35 is the device and 10.30.20.1 is the DHCP on the firewall.

In case you have not been able to glean from all my questions, I am not a firewall expert so I have not gotten very far in creating rules.  I have essentially gotten to the point that I have internet access for everything and that's about it.  I was wanting to just get everything working and then start fiddling with rules once I had the time.  I don't even know what the ACL is, so if there is one, it is not intentional and I did not put anything into it.

I will keep chasing this train of thought and see what I find.

Thanks for getting me pointed in the right direction.  Interestingly, the device that I am following has yet to fall off the network.  Previously it had only taken a few minutes, but we're over an hour here and it is still connected.  

I seem to always get the weird ones.

I have not created an ACL (that I know of at any rate).  Can you link to your thread just for fun?

Also, how is your DHCP set up for that segment? Do you have a Use Interface IP as Gateway checked? Is your subnet mask set properly?

I'm thinking that somehow the XG is answering the initial broadcast in such a way that the device is attempting to renew at the wrong address. The XG has an ACL, I now see, but DHCP is not one of the checkboxes that allow you to allow or disallow access. But it could be a DHCP server setting for that segment that's slightly wrong but that basically works for more sophisticated clients.

(You don't happen to be using v19? There are a lot more rabbit holes to go down with v19's additional potential flags.)

Also, how is your DHCP set up for that segment? Do you have a Use Interface IP as Gateway checked? Is your subnet mask set properly?

I'm thinking that somehow the XG is answering the initial broadcast in such a way that the device is attempting to renew at the wrong address. The XG has an ACL, I now see, but DHCP is not one of the checkboxes that allow you to allow or disallow access. But it could be a DHCP server setting for that segment that's slightly wrong but that basically works for more sophisticated clients.

(You don't happen to be using v19? There are a lot more rabbit holes to go down with v19's additional potential flags.)

Alright, I was afraid that it would come to questions about this.  Yes, I have the 'use interface as gateway' checked.  This was my first stab at VLANS and it took me about a week to acquire enough understanding to actually get it working.  The AP is physically connected to a managed (well semi-managed) switch.  The AP has 3 VLANs and the default VLAN.  The switch is set up with the same VLANs so that all the traffic coming from the AP's, through the switch and on to the firewall is tagged.  This particular VLAN has its own DHCP as well as the physical port that it is layered on.

On the hardware, I have 5 ports.  Port 5 is my WAN.  Port 2 is my IoT and the physical port has a DHCP range of 10.10.20.1 - 254.  The VLAN for my IoT has a DHCP range of 10.30.20.1 - 254.

The AP's are physically wired to the managed switch.  Both AP's and the switch are on the 10.10.10.x segment and have a trunk to Port 1.  That same trunk has the WLAN VLAN and the physical LAN (10.40.10.1 and 10.10.10.1 respectively).

I've been concerned that there was something going on with all this VLAN business that was not right.  It really took me several days to get it all to work correctly.  What I'll say is that everything receives IP addresses like they should and the network is segmented as I expected it to be.

I am not on version 19.  My version is SFVH (SFOS 18.5.3 MR-3-Build408).

I will dig around in the ACL stuff to see what I can see.

And you have Sophos DHCP service set up for each VLAN, separately, right?

I do something similar: I have a Sophos AP and it has three SSIDs: my high-trust one is bridged together with two ports to create my LAN. The other two are on VLANs. In addition, I have another port that is dedicated to VOIP. So I have four DHCP servers configured -- and the VLAN ones use the VLAN as the interface. And I do have Use Interface IP As Gateway checked (but Accept Client Request via Relay is NOT checked).

The subnet mask is appropriately set. Pretty much probably what you have. And I, too, use static assignments for everything that's supposed to be on the subnet. The only time the dynamic range should be used is either on the Guest network or for a new device, to make it easier for me to get its MAC and give it a static address.

I'm using v19 and there is a section at the bottom that lets you set a boatload of potential DHCP flags. Most of them are to tell the DHCP clients about old-school services that no one uses anymore. There are settings to tell them the NTP clock offset, or things about DHCP renewals and leases that might be helpful to you. Don't move to v19 until you're ready of course. (I was running the EAPs and I like the improvements.)

The only thing explicitly labeled ACL that I can see is in Administration > Device Access, but as I mentioned it does not have DHCP as one of its checkboxes, so I think there's a broader ACL of which Device Access exposes most of them. And the only thing I have turned on for all the VLANS is DNS, but your problem isn't DNS. Dynamic Routing is off for every zone.

I really think it boils down to either: a) there's something slightly misconfigured such that your device thinks the reply to its initial broadcast is coming from a port that's not actually running a DHCP server and so connectivity is refused by the "ACL" which has a Deny unless a DHCP server is running there, or b) there some other setting somewhere such that your device is being rejected from contacting the DHCP server via a unicast (as opposed to broadcast) packet. Good luck!

(Again, there's the slight chance that your devices are remembering something from the previous DHCP server and you need to hard recycle them to factory settings or something so they learn what they need to learn about the new DHCP setup.)

Yes, each physical port has a DHCP of its own as well as each VLAN for a total of 7 DHCP servers.  I originally had 5 SSID's, but I have reduced it to 3.  I consolidated my cameras and IoT into one and eliminated the one I had set up for work stuff since the Meraki handles all that now.

We are kindred spirits on the static/dynamic thing.  Dynamic is only for new stuff in order to easily get the MAC address.

I can also see a whole bunch of DHCP options but I do not know what any of that stuff is for.  Maybe for another time in my life......

My device access is likely set the same as yours except that I have not created any zones specifically for my VLANs yet.  I want to so that I can more tightly control what those devices can do, but I have to get everything working right before I go trying to break it.  I am using LAN, WAN, DMZ (for the work Meraki), and VPN zones.  All the VLANs currently run on the LAN zone until I get it all working properly.  Then I'll move the VLANs to their own zone and start monkeying with tighter controls.

I think your assessment is correct here.  I finally got the device that I was packet capturing to drop off the network - it took like 5 hours which was very odd.  There was nothing interesting in the packet capture, it just stopped picking up packets to capture and that was it.  I ended up with 179 pages of packets which I think is way too much, but it is probably the device talking to the cloud to provide status, the cloud saying 'ok, got it' and then the violation message.  I got one of those every time the lease expired, but the device stayed connected for a very long time.  There was also a recurring message from the device to DHCP and back where the packet type was ARP - NDP request.  The first message is always from the device to DHCP and the status is Consumed.  The return message status is always Generated.  Not real sure what any of that means, but I thought it was noteworthy.

I am working on setting the old AP up to test it and I am setting up virgin devices - one on the new AP and one on the old AP just for fun.

Fingers crossed.

By the way, thanks a ton for your input.  Not many folks left who like to fool with amateurs on this kind of stuff and you have certainly got me farther than I likely would have on my own.