12 Replies Latest reply on Sep 19, 2017 5:22 AM by Silly-V

# How can I read the color of a given pixel in an image?

Hi all,

I'm trying to find a way to read the color of certain pixels in a raster image in Illustrator through scripting. I was hoping there would a sort of getPixel function that would read out the value of a pixel's color based on the pixel's XY-coordinates in the image but I'm not finding any info in the reference guide on this.

Is there a way to do this in Illustrator?

• ###### 1. Re: How can I read the color of a given pixel in an image?

I think you can clip off the pixel you want from the image and run the live trace to get the color as Illustrator would get it using the trace feature. This is extreme, and probably doing a script which opens the image in photoshop and uses photoshop scripting to read the pixel, then return the result to AI is better.

I think it would be interesting to do the Photoshop exercise, some day. So how will you determine the XY coordinate? If your placed image is scaled down, while the original is is larger, is the XY going to be defined by something in the AI document?

• ###### 2. Re: How can I read the color of a given pixel in an image?

Regarding the XY-coordinate, there are many ways to determine these, all pretty easy to use with some simple linear mathematics: many frameworks allow a command like myColorVariable = myImage.getPixel(x,y) with x and y expressed either in a floating point value between 0 and 1 (spanning the width and height of the image) or as an integer representing the specific pixel. Basic stuff really, with the function getPixel() returning the values for color channels and alpha. Unfortunately there does not seem to be a way to do this in Illustrator using scripting, which is a bit mindblowing given that it's 2017 by now!

Come on Adobe! We paying plenty for CC now, supposedly so development is swift and strong. This is a no-brainer that should be in the scripting in my opinion.

Clipping the image seems to be a very slow and cumbersome way of doing this - and passing through Photoshop also seems a bit overcomplicated, but: thanks for the suggestions! At least you do offer workaround that would do the job!

• ###### 3. Re: How can I read the color of a given pixel in an image?

Hmmm. There is no "clip" method for rasterItems. That means I probably need to start creating clipping masks myself, duplicating the image, rasterizing the clipped part, and then tracing that pixels.

FOR EACH PIXEL!!! I'm open to using workarounds, but this is a bit too backwards for my taste. I'll see if the Photoshop trick could work...

• ###### 4. Re: How can I read the color of a given pixel in an image?

Well, this is only for test. It use object masic. If the image size(pixels) is bigger than 30*30 or 50*50, it will become very slow. Note: the image should be 72 dpi.

So step 1: make sure image is 72 dpi, if not, reset it(this can be done with script);

step 2: crop(not clip) the image to size like 10*10 px (contain the given pixel of cause. Not sure this can be done with script or not.)

Anyway, here is the code.

```String.prototype.formatUnicorn = function() {
"use strict";
var str = this.toString();
if (arguments.length) {
var t = typeof arguments[0];
var key;
var args = ("string" === t || "number" === t) ?
Array.prototype.slice.call(arguments) :
arguments[0];
for (key in args) {
str = str.replace(new RegExp("\\{" + key + "\\}", "gi"), args[key]);
}
}
return str;
};

function objectMasic(w, h) {
var actionStr = '''
/name [ 1    73]
/actionCount 1
/action-1 {
/name [ 1    61]
/eventCount 1
/event-1 {
/internalName (ai_plugin_mosaic)
/hasDialog 1
/showDialog 0
/parameterCount 7
/parameter-1 {
/key 1937208424
/type (integer)
/value 0
}
/parameter-2 {
/key 1936222068
/type (integer)
/value 0
}
/parameter-3 {
/key 1953985640
/type (integer)
/value {width}
}
/parameter-4 {
/key 1952999284
/type (integer)
/value {height}
}
/parameter-5 {
/key 1668246642
/type (boolean)
/value 1
}
/parameter-6 {
/key 1684368500
/type (boolean)
/value 0
}
/parameter-7 {
/key 1886544756
/type (boolean)
/value 0
}
}
}
'''.formatUnicorn({width:w, height:h});
createAction(actionStr, 's');
app.doScript('a', 's');
}

function createAction(str, set) {
var f = File(set + '.aia');
f.open('w');
f.write(str);
f.close();
f.remove();
}

function getPixel(image, x, y) {
var h = image.height.toFixed(0),
w = image.width.toFixed(0);
objectMasic(w, h);
return app.selection[0].pathItems[w * (y - 1) + x - 1].fillColor
}

var color = getPixel(app.selection[0], 3, 3);
app.selection[0].remove();
```
• ###### 5. Re: How can I read the color of a given pixel in an image?

Wow - I'm mainly a Lua programmer, so I'll be honest. I don't understand much of what's going on there! Is that javaScript?

What type of variable is this:

var actionStr = '''

with all the slashes in there... Wondering what type of variable that is... And what ai_plugin_mosaic is!

Either way, I was looking for a scriptoGrapher replacement, unfortunately to use to create thousands of images, based on thousands of pixels. So while your code is very cool, it won't do me much good for now.

But still, thanks for helping out - and I wish I could read your code better to see what's going on...

• ###### 6. Re: How can I read the color of a given pixel in an image?

For clarity: the above post is mine, the OP. A colleague was still signed in on my laptop, hence the confusion.

• ###### 7. Re: How can I read the color of a given pixel in an image?

Raster items can be clipped by being moved into a group item, and having a path in that group item have its property .clipping set to true and having that group item also have its property .clipped set to true.

Also you can make a path, select it and your target item, and use a menu-command execution ( app.executeMenuCommand("makeMask"); ) to clip the item more automatically.

As for the photoshop idea, you can start out in Illustrator and use a scripting object 'BridgeTalk' to open a referenced image in photoshop, get the information and use it back in your Illustrator routine. Since traveling entire applications to get the information, perhaps you can get all the information and save it in a tag of pageItem.tags property just the first time and then use other functions to read back this information.

Looks like the thread back from 2009 here mentions obtaining color at coordinates:  JavaScript: Get color of a defined Pixel

The code above has the one-and-only Illustrator Action "language", it's what you get when you record an Illustrator action and save the action sets in a .aia file. Learn more here.

• ###### 8. Re: How can I read the color of a given pixel in an image?

As I know we can use python to scripting Illustrator, and with PyPNG it's easy to get the pixels info of a PNG file. Don't know if lua is the same.

• ###### 9. Re: How can I read the color of a given pixel in an image?

I'm making a bt example and it appears to be working! I'll post it when it's fully ready.

• ###### 10. Re: How can I read the color of a given pixel in an image?

I finally was able to produce this code, which I will undoubtedly I will refer to for my own practices in the future! Questions/Comments/Improvements are totally welcome.

```#target illustrator
/* without targetengine here, this script does nothing when ran from Illustrator & not ESTK */
#targetengine main
/* script function definition with argument specified */
function getAIPixelInfoViaPhotoshop(argsObj){
/* JSON snippet pasted in from internet */
"object"!=typeof JSON&&(JSON={}),function(){"use strict";function f(t){return 10>t?"0"+t:t}function quote(t){
return escapable.lastIndex=0,escapable.test(t)?'"'+t.replace(escapable,function(t){var e=meta[t];
return"string"==typeof e?e:"\\u"+("0000"+t.charCodeAt(0).toString(16)).slice(-4)})+'"':'"'+t+'"'}
function str(t,e){var n,r,o,f,u,i=gap,p=e[t];switch(p&&"object"==typeof p&&"function"==typeof p.toJSON&&(p=p.toJSON(t)),
"function"==typeof rep&&(p=rep.call(e,t,p)),typeof p){case"string":return quote(p);case"number":return isFinite(p)?String(p):"null";
case"boolean":case"null":return String(p);case"object":if(!p)return"null";if(gap+=indent,u=[],"[object Array]"===Object.prototype.toString.apply(p)){
for(f=p.length,n=0;f>n;n+=1)u[n]=str(n,p)||"null";return o=0===u.length?"[]":gap?"[\n"+gap+u.join(",\n"+gap)+"\n"+i+"]":"["+u.join(",")+"]",gap=i,o}
if(rep&&"object"==typeof rep)for(f=rep.length,n=0;f>n;n+=1)"string"==typeof rep[n]&&(r=rep[n],o=str(r,p),o&&u.push(quote(r)+(gap?": ":":")+o));
else for(r in p)Object.prototype.hasOwnProperty.call(p,r)&&(o=str(r,p),o&&u.push(quote(r)+(gap?": ":":")+o));return o=0===u.length?"{}":gap?"{\n"+gap+
u.join(",\n"+gap)+"\n"+i+"}":"{"+u.join(",")+"}",gap=i,o}}"function"!=typeof Date.prototype.toJSON&&(Date.prototype.toJSON=function(){
return isFinite(this.valueOf())?this.getUTCFullYear()+"-"+f(this.getUTCMonth()+1)+"-"+f(this.getUTCDate())+"T"+f(this.getUTCHours())+":"+
f(this.getUTCMinutes())+":"+f(this.getUTCSeconds())+"Z":null},String.prototype.toJSON=Number.prototype.toJSON=Boolean.prototype.toJSON=function(){
return this.valueOf()});var cx,escapable,gap,indent,meta,rep;"function"!=typeof JSON.stringify&&
meta={"\b":"\\b","  ":"\\t","\n":"\\n","\f":"\\f","\r":"\\r",'"':'\\"',"\\":"\\\\"},JSON.stringify=function(t,e,n){var r;
if(gap="",indent="","number"==typeof n)for(r=0;n>r;r+=1)indent+=" ";else"string"==typeof n&&(indent=n);if(rep=e,
e&&"function"!=typeof e&&("object"!=typeof e||"number"!=typeof e.length))throw new Error("JSON.stringify");return str("",{"":t})}),
JSON.parse=function(text,reviver){function walk(t,e){var n,r,o=t[e];if(o&&"object"==typeof o)for(n in o)Object.prototype.hasOwnProperty.call(o,n)&&
(r=walk(o,n),void 0!==r?o[n]=r:delete o[n]);return reviver.call(t,e,o)}var j;if(text=String(text),cx.lastIndex=0,cx.test(text)&&
(text=text.replace(cx,function(t){return"\\u"+("0000"+t.charCodeAt(0).toString(16)).slice(-4)})),
/^[\],:{}\s]*\$/.test(text.replace(/\\(?:["\\\/bfnrt]|u[0-9a-fA-F]{4})/g,"@")
.replace(/"[^"\\\n\r]*"|true|false|null|-?\d+(?:\.\d*)?(?:[eE][+\-]?\d+)?/g,"]")
.replace(/(?:^|:|,)(?:\s*\[)+/g,"")))return j=eval("("+text+")"),"function"==typeof reviver?walk({"":j},""):j;
throw new SyntaxError("JSON.parse")})}();
//================================================================== BRIDGETALK ==================================================================//
function bridgeTalkEncode(txt) {
/* thanks to Bob Stucky */
txt = encodeURIComponent(txt);
txt = txt.replace(/\r/, "%0d");
txt = txt.replace(/\n/, "%0a");
txt = txt.replace(/\\/, "%5c");
txt = txt.replace(/'/g, "%27");
return txt.replace(/"/g, "%22");
};

function bridgeTalkDecode(txt) {
return txt.replace(/%0d/g, "\r").replace(/%0a/g, "\n").replace(/%5c/g, "\\").replace(/%27/g, "'").replace(/%22/g, '"');
};

/* BridgeTalk messaging function which can send entire script with arguments to a target application, and retrieve the result */
function sendBTmsgArgs(func, argsObj, targetApp, resultFunc) {
var bt = new BridgeTalk;
bt.target = targetApp;

var functionName = (func.name == "anonymous") ? "" : func.name;
var startRx = new RegExp("^\\(function\\s*" + functionName + "\\(\\)\\{");
var btMsg = func.toString();

var meat;
if (typeof argsObj != "undefined") {
meat = btMsg + ";\n" + functionName + "('" + JSON.stringify(argsObj) + "');";
} else {
meat = btMsg + ";\n" + functionName + "();";
}

meat = bridgeTalkEncode(meat);

btMsg = "var scp ='" + meat + "'";
btMsg += ";\nvar scpDecoded = decodeURI( scp );\n";

if (typeof resultFunc == "function") {
btMsg += "btResult = eval( scpDecoded );";
} else {
btMsg += "eval( scpDecoded );";
}

bt.body = btMsg;
if (typeof resultFunc == "function") {
bt.onResult = function(info) {
info = JSON.parse(info.body);
resultFunc(info);
};
}

bt.onError = function(res) {
};

bt.send();
};
//================================================================= /BRIDGETALK ==================================================================//

//==================================================================== COMMON ====================================================================//
/* credit for this algorithm goes to Dan Schiffman of The Coding Train on youtube. */
function overlap(item1, item2) {
var l1 = item1.left,
t1 = item1.top,
r1 = item1.left + item1.width,
b1 = item1.top - item1.height;

var l2 = item2.left,
t2 = item2.top,
r2 = item2.left + item2.width,
b2 = item2.top - item2.height;
return !(l1 > r2 || t1 < b2 || b1 > t2 || r1 < l2);
};
//==================================================================== /COMMON ====================================================================//

//====================================================================== BRIDGE ===================================================================//
function mainBr(argsObj) {
var storedCurrentBridgePath = app.document.presentationPath;

var fileList = [],
counter = 0;
while (fileList.length == 0 && counter++ < 50) {
app.document.thumbnail = File(argsObj.testFilePath).parent;
var thumbnails = app.document.thumbnail.children;
for (var i = 0; i < thumbnails.length; i++) {
fileList.push(thumbnails[i].name);
}
BridgeTalk.bringToFront("bridge"); /* this was absolutely necessary because sometimes bridge document would not be updated, especially after Bridge restarts */
}
app.document.thumbnail = storedCurrentBridgePath;
return JSON.stringify({
bridgeFileList: fileList
});
};
//===================================================================== /BRIDGE ===================================================================//

//==================================================================== PHOTOSHOP ==================================================================//
function getPixelInfo(x, y) {
/* function to use a colorSampler to sample color at a specified pixel, returns color information based on a subset of available document modes */
var doc = app.activeDocument;
for (var i = 0; i < doc.colorSamplers.length; i++) {
doc.colorSamplers[i].remove();
}
try {
sampler.color;
} catch (e) {
return "transparent";
}
var result, rounding = 0;
if (doc.mode == DocumentMode.CMYK) {
result = "cmyk(" + (sampler.color.cmyk.cyan).toFixed(rounding) + ", " + sampler.color.cmyk.magenta.toFixed(rounding) + ", " + sampler.color.cmyk.yellow.toFixed(rounding) + ", " + sampler.color.cmyk.black.toFixed(rounding) + ")";
} else if (doc.mode == DocumentMode.RGB) {
result = "rgb(" + sampler.color.rgb.red.toFixed(rounding) + ", " + sampler.color.rgb.green.toFixed(rounding) + ", " + sampler.color.rgb.blue.toFixed(rounding) + ")";
} else if (doc.mode == DocumentMode.GRAYSCALE) {
result = "grayscale(" + sampler.color.gray.gray.toFixed(rounding) + ")";
} else {
alert(doc.mode + ": getPixelInfo not implemented");
result = null;
}
doc.colorSamplers[0].remove();
return result;
};

function getPixelInfoFromImage(x, y, w, h, srcFile) {
/* function which opens a specified document, and returns pixel color information from a pixel coordinate set */
var desc = new ActionDescriptor();
desc.putPath(charIDToTypeID("null"), new File(srcFile));
executeAction(charIDToTypeID("Opn "), desc, DialogModes.NO);
var doc = app.activeDocument;
x *= (doc.width / w);
y *= (doc.height / h);
var result = getPixelInfo(x, y);
app.activeDocument.close(SaveOptions.DONOTSAVECHANGES);
return result;
};

function mainPs(argsObj) {
/* the main photoshop script,  */
var data = getPixelInfoFromImage(argsObj.coords[0], argsObj.coords[1], argsObj.w, argsObj.h, argsObj.testFilePath);
return JSON.stringify({
data: data,
testFilePath: argsObj.testFilePath
});
};
//=================================================================== /PHOTOSHOP ==================================================================//

//================================================================== ILLUSTRATOR ==================================================================//
function Ai_Script_2(btResult) {
/* [STEP (7)] back in Illustrator, returned data from last BridgeTalk call, to which this function was attached as onResult, is called */
/* [STEP (8)] to test another BT call to another app, this function targets bridge and sets the next onResult Illustrator function */
sendBTmsgArgs(
getAIPixelInfoViaPhotoshop, {
launchScript: "mainBr",
testFilePath: btResult.testFilePath
},
"bridge",
Ai_Script_3
);
};

function Ai_Script_3(btResult) {
/* [STEP (10)] using data back from Bridge to show a list of neighboring files which Bridge found */
alert("Bridge looked at placed image's folder. Other files in that folder are:\r" + btResult.bridgeFileList.join("\n"));
};

function mainAi() {
/* [STEP (3)] do a process in Illustrator */
if (app.documents.length < 1) {
return;
}
var doc = app.activeDocument;
if (doc.selection == null || doc.selection.length != 1) {
return;
}
var sel = doc.selection[0];
if (!sel.hasOwnProperty("file")) {
return;
}
var testFilePath = decodeURI(sel.file);
var sourcePoint, coords, testCoords;
try {
sourcePoint = doc.pathItems.getByName("Source Point");
coords = [
((sourcePoint.left - sel.left) + sourcePoint.width / 2), -((sourcePoint.top - sel.top) - sourcePoint.height / 2)
];
testCoords = [
((sourcePoint.left) + sourcePoint.width / 2),
((sourcePoint.top) - sourcePoint.height / 2)
];
} catch (e) {
alert("Use a circle named 'Source Point' to mark the pixel source area by its center");
return;
}
/* check to ensure the chosen location in Illustrator matches a point on top of a placed item */
if (!overlap(sel, {
left: testCoords[0],
top: testCoords[1],
width: 0,
height: 0
})) {
alert("Make sure center of 'Source Point' circle is on top of the selected item's area.");
return;
}
var argsObj = {
launchScript: "mainPs",
testFilePath: testFilePath,
coords: coords,
w: sel.width,
h: sel.height
};
/* [STEP (4)] send a script to photoshop along with arguments and the name of the function to play after BridgeTalk message returns */
sendBTmsgArgs(getAIPixelInfoViaPhotoshop, argsObj, "photoshop", Ai_Script_2);
};
//================================================================= /ILLUSTRATOR ==================================================================//

/* "entry block": this conditional statement determines which parts of this entire script are to be executed */
if (typeof(argsObj) != "undefined") {
try {
argsObj = JSON.parse(argsObj);
} catch (e) {
return;
}
if (argsObj.launchScript == "mainPs") {
/* [STEP (5)] this entire script has been passed into Photoshop, and this argument decides which part is executed
(this helps with code re-use, even among common functions used by different applications) */
return mainPs(argsObj);
/* [STEP (6)] this return will output the data needed to be used by the onResult function attached to the last BridgeTalk message */
} else if (argsObj.launchScript == "mainBr") {
/* [STEP (9)] execute the Bridge portion of this script, return the data so an onResult function can use it */
return mainBr(argsObj);
} else {
return;
}
} else {
/* [STEP (2)] call the function which starts the process from Illustrator */
mainAi();
}
};
/* [STEP (1)] script is initially activated with no argument */
getAIPixelInfoViaPhotoshop();
```

What this is, is a demonstration of a cross-application script. It starts out in AI, where initial conditions need to be an open document with image files, one of which has to be selected, and one circle pathItem named "Source Point". The AI portion will get the coordinates of this circle in relation to the selected placed item and send the information in a bridgetalk message to photoshop. Photoshop opens the document, reads the pixel information at the given coordinates proportionate to the placed item's width/height compared to the open document's full size. Then it sends the pixel information back and an AI alert message alerts the result. The last demonstration step looks up the selected placed file's folder and enters it in Bridge. It gathers the file names of all the files in this folder and again sends a BridgeTalk message back to illustrator where another onResult fuction deals with the data by alerting it.

This method is used to work on placed files, not raster items which are embedded. Getting pixel information is a statement with many meanings, when talking about placed items because of the variety Adobe allows. For example, the image could be in another color mode than RGB. Illustrator RGB documents can have CMYK placed images and vice versa, and, the placed file may have whole other modes too.

Other ways to get the pixel information, when opening up the exact file is not necessary, could be to export art as PNG using export-selection scripting commands.

One point about BridgeTalk I learned is that it works for interaction between the main script's app and one target app at a time. Originally I thought that a bridgetalk script could send other bridgetalk scripts at the end of their code body, to another application, going on indefinitely. However, this produced a faulty result where both apps were doing the same code, and the target never switched, all kinds of issues. Switching to the proper way of using onResult functions works though, and JSON strings can be transmitted to pass complex objects.

In going between different applications it amounts to starting in Illustrator, performing other app action, going back to Illustrator, then performing another other app action and so on. So if it for some reason needed to go to an Indesign script from the photoshop script and use that Indesign result in the rest of the photoshop script, it would have to be split into 2 photoshop functions and the indesign function to be executed between them, from Illustrator.

I had some difficulty when Bridge would sometimes not have its view refreshed after my code opened a folder in it - I think I solved it by putting in a while() loop with a max counter.

All in all, being able to use cross-application scripting has great potential, but it truly depends on the needs of the workflow. In this example it's simply reading a pixel and alerting folder names, but in theory photoshop could be used to do some more advanced color detection and bridge (with it's soon to be fixed webaccesslib) could interact with a web API.

*NOTE:

In this example, I am using alerts to show that data was transmitted among applications. However, back in Illustrator, I am not actually using any DOM methods to speak of. One would assume you just keep on working in AI as usual, once inside the onResult block. This is not the case: the AI DOM completely disappears out of reach after the first BridgeTalk message is passed to the first application. The solution to this would be to then on treat Illustrator as yet another BT target - except do not what I tried to initially do, which is to keep sending nested BT messages at the end of containing BT message bodies. The BT message to Illustrator should be send inside the onResult block, and then it will work.

• ###### 11. Re: How can I read the color of a given pixel in an image?

impressive, great work, thanks for sharing sv

• ###### 12. Re: How can I read the color of a given pixel in an image?

Oh no, and terrible news: apparently BridgeTalk just doesn't work with Illustrator Actions Batch process.

From my tests it looks like the batch actions keep playing without waiting on the menu-item-inserted script result, if that script does a BT action. I was trying to revisit my test of making QR code using Indesign: the ID script will make the QR code and do an app.copy(), then in the result function sent to AI, the AI script does an app.paste();. Well, it kept pasting in too late - well after the next dataset's action started playing. However, I was able to use the Pause for: mode in the Playback Options and increase the pause to 2 seconds. This allowed enough time to have the BT script paste in the QR paths. This is far from ideal and is excruciatingly slow, and the only think I can think of to make it sync is making code to watch for creation/destruction of a marker text file.