You are not alone – http://gondwanaland.com/mlog/2007/09/11/passwordless/

I presume the system you envisage goes like this:

1) “Click here to log in via a URL e-mailed to you”
2) Wait for e-mail
3) Click URL in e-mail
4) Now logged in (via other window)
5) Refresh/visit other page to find one is now logged in

Sounds like a reasonable idea and probably convenient for those with rapid e-mail delivery.