Example font subset with non-unicode output

I don't really understand the question. ToUnicode is an add-on for text extraction. It has no connection to the other font parameters, and if it is removed it will not affect PDF display in any way. It will only reduce the likelihood of good text extraction.

It also does not mean that the font, or the text to display, is in UTF-16 encoding. The font as declared will have text data in the page stream containing CIDs, not Unicode.

Message was edited by: Test Screen Name

Sorry, I don't see what your requirement has to do with ToUnicode.

You would not be using the font's "cmap" table to display, you would use a "CMap" and these are not the same thing and do not map. You would not code UCS2 data in the page stream. You would be using Identity-H and encoding the data as CIDs (which is the same thing as GIDs in the font).

You appear to be mixing up a few things - which is why font subsetting in PDF is hard.

There is the original font encodings (as found in cmaps, for example).

There is the encoding of the subset data itself (which for CID fonts is almost always custom encoding)

There is what you put in you "draw string" commands.

In the process of subsetting, the embedded data is custom encoded and the values written int he draw string matches that encoding (and looks nothing like a standard encoding).   For example, if you have a subset with 'a', 'b', 'c' and 'd' using ID's 1-4 respectively, and you wanted to draw the string 'bad', then you would write

(\2\1\4)Tj

As described in ISO 32000-1:2008

Without the actual font data - no idea what's wrong. You only provided a very limited bit of PDF.

I don't know whether the font data is valid or not, but you certainly are not passing CIDs to Tj. You are just passing 1-byte ANSI text.

Why not just use CIDToGIDMap Identity, and use CIDs. You seem determined to make the font fit what you want to put in Tj, rather than put in Tj what works with the font!

Stop trying to use ANSI characters in Tj. What makes you believe this is correct or valid?

And by the way, that code you posted in reply #14 looks a good start for working with CIDs/GIDs. What makes you guess it is Unicode? I Think you are perhaps focussing too much on Unicode; it is almost completely irrelevant to the task of writing the PDF, and Unicode is almost never written to make streams.

The byte values in a string are what the font requires, according to its Encoding (1-byte fonts) and combination of CMap/internal CIDs (2-byte fonts). Often you will see for 1-byte fonts ANSI characters, because the Encoding used happens to be WinAnsiEncoding or MacRomanEncoding, which are the same as ANSI for byte values < 127. But, so what? You are not writing a 1-byte font.

"So when I now use a font, which does not provide Unicode (like Arial - NOT ARIALUNI) why I should convert it into UTF-16BE?"

You should not. Why would you, and why do you think that Unicode is involved? Are you just assuming that when you see a 2-byte value that it is 2-byte Unicode? Never assume, read the spec!!

Just to add to this excellent explanation.

A CID font is a TWO BYTE FONT.  So by choosing to use one, you are requiring the use of two bytes per glyphs.

If you only want a single byte per glyph, use a simple TrueType font with a standard encoding (such as WinANSI).

Text encoding in PDF in essence is for glyph lookup only.

On one end of a processing chain, you have a character code in a text showing operator.

On the other hand you have a glyph in a font.

PDF (and by reference several font specifications) defines several ways how to define (in the code of a PDF file) how certain character codes are - for a given font resource - linked to  the font resource and the glyphs in it. As there are a number of ways to do this, it can be very confusing for the novice. In order to be successful in this field you must fully understand the part of the PDF specification/standard that defines how to put text on a PDF page. If there are parts you do not understand yet - keep reading the spec until you understand it. There is no silver bullet or magic trick to do text in PDF without this.

It might be true, under certain circumstances, and in many cases coincidentally so, that character codes happen to be  equivalent to single byte text codes (as defined WinANSI, MacRoman, MacExpert encodings, and in addition through a Differences array and use of predefined glyph names as defined by AGL). But it is not smart to hope for this, especially as any of this could - for the purpose of text extraction - be overwritten by a ToUnicode table (and a character that accoridng to WinANSI represents "A" would have to be extracted as "B" if the ToUnicode table says so).

So in the interest of writing better code, forget about the notion that character codes have in any deterministic anything to do with text / Unicode code codepoints, single byte or not. Character codes are just numbers that let you look up a glyph in a font if a certain procedure is followed...

Try not to use or think the word "Unicode" or Unicode code points until you have managed to understand how PDF defines the relationship between character code and glyphs. Unicode does not have any role here. FirstChar and LastChar related only to the character codes in a Simple Font (where character codes have one byte). I have no idea what you mean by FirstCharacter and LastCharacter (try hard not to be sloppy with keywords!)

I think you do have a lot of reading ahead  of you - just grab the PDF spec and read the part on text again. Try to focus on one approach (Simple Font / TrueType in your case), until you've got it.

No, this is proceeding from wrong assumptions.

For all single byte fonts, EVERY PDF contruction deals with 256 values, range 0 to 255.

Let's look at what you say. You mention two Unicode values, and you continue to think that Unicode has some importance or meaning in PDF. No, it doesn't. Perhaps your characters are U+0021 and U+FEFC but the PDF does not care, and nor should you (when writing the PDF). I emphasise: if you are thinking "I will put this Unicode value somewhere in the PDF" and you are dealing with page streams/fonts, you are almost certainly not following the spec.

For a single byte font the mapping from the arbitrary codes in the PDF to the internal font structures has one and only one variable: the Encoding value. The Encoding value, if present (and the situations in which it can be omitted are specific and exact), is a shorthand representation for something which will always be an array of 256 names.

Since TrueType fonts don't contain names, the interpretation of this is complicated for embedded TrueType, and you must take care to follow exactly what the spec says. Other things MAY SOMETIMES WORK without being valid, but will probably fail in different software, so stick to the spec exactly.

There is no substitute for endless, deep and repetitive study of the PDF Reference. It will help if you can phrase your future questions in terms of "what does it mean in the specification when it says...". That will help keep you focussed. I have been working with the PDF specification for many, many years and I have found that whenever I assume anything, I am almost always wrong. Read the spec first, second and last.

FInally, don't be discouraged if you find this hard. Font embedding IS one of the very hardest things to master about PDFs, easily adding an order of magnitude of complexity to any project.

Shortcut for 1-byte fonts.

All of these are1-byte value with the same meaning

- FirstChar, LastChar

- Indexes in WIdths (offset by FirstChar)

- values in the byte strings passed to Tj etc.

- indexes into the 256 name array generated from Encoding

None applies to Type 0/multibyte fonts.