If my ex doesnt get our son on her scheduled visitation day can i legally keep him until her next visitation?

hi so my ex wife doesnt call or show up to pick up my son most of the time, so can i tell her she has to wait until its her next actual scheduled day to pick him up or do i have to allow her to be able to pick him up (example, she is supposed to get him on friday and she wont call to get him till sunday). and how often can that happen before it is considered child abandonment oh her part?