Anthony was survived by his wife, Leyin Ouyang, and his son and daughters, Frank, Diane, Ellen, Deborah, and Juliann. He was predeceased by his mother, Aida Scanio Castelbuono, and his father, Frank A. Castelbuono. He had no children with Leyin Ouyang, though they have had five dogs throughout their 25 years of marriage.
Anthony was the single child of Frank and Aida Castelbuono.
Anthony went to and graduated from Regis High School, Fordham University, and Harvard Law School. He lived in New York, New York; South Orange, New Jersey; Henderson, Nevada; and Las Vegas, Nevada.