Xprobe2 : Active OS fingerprinting tool
XProbe is a tool for determining the operating system of a remote host. They do this using some of the same techniques as Nmap as well as some of their own ideas. Xprobe has always emphasized the ICMP protocol in its fingerprinting approach.
P0f : A versatile passive OS fingerprinting tool
P0f is able to identify the operating system of a target host simply by examining captured packets even when the device in question is behind an overzealous packet firewall. P0f does not generate ANY additional network traffic, direct or indirect. No name lookups, no mysterious probes, no ARIN queries, nothing. In the hands of advanced users, P0f can detect firewall presence, NAT use, existence of load balancers, and more!