Thinking about

What a mary sue is depends a lot on the story, but here's a few things that can help:
1) First and foremost be willing to listen to critique once you've made the characters, which I will assume should be no problem since you're asking for help up front.
2) Think about what the story revolves around: Does it revolve around a character or is the character just playing a role in the story?  If a certain character is always in the middle of everything, even when it doesn't make sense for them to be there, that's a bad sign.  Another is if things happen in the story just to create drama for a character rather than to make an interesting plotline.
3) Another type of sue is one that revolves around a character; if a character only exists to be in a relationship with another character and cannot stand by themselves as an interesting character, and the revolving character is a major character, that's a bad sign.  Minor characters can get away with this, because maybe they only exist as a plot device (oh no!  X's girlfriend was kidnapped!) but even then, that character won't be one anyone cares about.
4) Be balanced.  People tend to think that giving a character one weakness makes them not a sue, but that's not how it works.  You want you character to have strengths and weaknesses, sure, but it's more about what are said strengths and weaknesses are and how they fit into the story.  If you make a sword fighter magician, that's what you should have: a sword fighter magician.  The character probably shouldn't also be a master linguist and know the deep ancient history of some unknown race and more things that are useful to the plot.  When it comes to your weaknesses, they have to actually be relevant and something you intend to use.  "My character is allergic to pickles" is a pointless weakness unless they will be forced to eat pickles or confront someone with pickles.
5) Adversity: you want neither a character that whines and bitches before having to do something they don't want to do just for the sake of angst/drama nor one that can one-hit everything (or a combination of the two).  The way you build tension is by having a real problem the main characters cannot easily overcome and that they may even lose against (at first or forever depending).  That means no whining for chapters about a lost boyfriend (though is can be a motivation brought up at critical points) and that means no "dropkicking the villain unconscious."
6) As I alluded to above: motivation.  Why does you character want to do what they are doing?  "They're just evil" is shitty motivation as is "for the sake of peace and love and purity!"  Your character needs a reason to feel this way even if it's as simple as "they don't get along" for a rivalry between two people or "he wants revenge" for justifying evil acts or "she's seen people suffer" for why a character is fighting for goodness.  Motivation is king.  Stories, from a character standpoint, are all about one character's motivation competing with or allying with another's.
7) A personality.  It's sad how many characters lack one.  Motivation is a part of this, but if the best you can describe your character as is "cute, kind, only fights bad guys, likes sweets," you haven't actually developed the character.  Is this an introvert or an extrovert?  Does this character plan or act on impulse?  Loud and obnoxious or shy and reserved? Does this person make decisions quickly, or do they really like to sit back and think things through?  Are they assertive or permissive?  Do they start fights or avoid them?  There are a ton of other questions like this and if you can't answer them, then you're character isn't developed enough.
8) SHOW don't tell.  It's great and all to hear someone is a great fighter, but if I don't see it, I don't believe it.  If you character is "kind" I want to see them being kind, but just be told about past exploits.  And I expect their actions to be consistent with their personality.  The evil vengeful villain should be metaphorically gunning down puppies, not running about saying they'll do bad things to annoy the main protag but never actually hurting anyone.

Basically, make a character that makes sense, has a personality and motivation, and that is not the center of the universe and you're off to a good start.