I had an affair with a guy I meet before my husband. Now this guy ask me to move with him. What should I do?

I been married for 12 years now (got married in love) and have two kids with him. Over the years this relationship has been wearing off. He is uncaring, not affectionate and inconsiderate. But a good man and a good father. This other guy I know him for over 16 years now, we where neighbors when I was 14 and always sort of like him and always try to get in contact with him over the years. I know him very well, we have a lot in common and I know the way he is and I really like his way. I really feel for him. I wanna be with him. My biggest concern is the kids. SHOULD I STAY FOR THE KIDS???