The Icelandic words sem and að can both translate as that in English, which makes it easy to get them mixed up. Let’s make it a little easier to separate the two.
Sem and að have a few different meanings, but where people tend to confuse them (and what we’ll be talking about here) is sem as a relative conjunction (the English equivalents, who, whom, that, and which, are usually categorised as relative pronouns) and að as a subordinate conjunction. Basically, here we’re talking about sem and að when they connect things.
The difference between the two words is quite simple. Basically, it’s like this:
sem follows nouns, while að follows verbs
Yes, this is a simplification, of course, but it’s a very useful one for now so just, like, don’t question it, okay? Both words can serve in different roles, but nobody mixes them up in those roles, so this is the difference that matters to us now.
Að halda (to think), að vita (to know), and að efa (to doubt) are all verbs, and thus they’re followed by að, not sem.
Konan (the woman), strákurinn (the boy), borðið (the table) and Myanmar are all nouns, and thus they’re followed by sem, not að.
As a side-note, isn’t it nice that Icelandic is super simple with just one word (sem) to English’s four (who, whom, that, which)?
Note that you cannot omit sem or að like you can that in English.
What you can do, though, is insert an að after sem. This is not unique to sem; you can do this after virtually any conjunction. It doesn’t change anything, but it’s considered non-standard. Avoid it in formal texts.