You have to turn on ACLs for this to work correctly, as you can no-longer turn on inherit permissions from parent to my knowledge like you could in 10.4.x
Also until 10.5.5 came out SMB permissions also failed to work correctly, giving you the same problem you mentioned in your post.