16 Replies Latest reply on Oct 13, 2006 5:24 AM by duckets

    Pitch recognition

    cratica Level 1
      Are there any samples or plugins available that uses pitch recognition? I was to take a mic input and output in real time the pitch or key of the voice (karaoke). I'm rather desperate and have looked extensively on the net for something that will work including taking some of the plugins that lets you get the Fast Fourier Transform (frequency) and trying to decode this into a pitch. Nothing is working. I want to create a karaoke game that matches the mic input to a known key.

      Please let me know if there is any existing xtras or if anyone is willing to create one and how much it would cost. Thanks!!!
        • 1. Re: Pitch recognition
          duckets Level 1

          Maybe Antoine Schmitt's "asFFT Xtra" would help you?

          http://www.as-ci.net/asFFTXtra/

          hope this helps!

          - Ben
          • 2. Re: Pitch recognition
            cratica Level 1
            Thanks but as I mentioned in my initial post, those plugins don't work. I tried that asFFT as well as other xtras that get just the frequency. The frequency isn't the same as the pitch of a voice. I don't see a way to match the frequency from aaFFT to a known key or pitch.

            Anybody?
            • 3. Re: Pitch recognition
              Level 7
              On Mon, 09 Oct 2006 23:09:27 GMT, cratica posted in
              macromedia.director.lingo:

              > Thanks but as I mentioned in my initial post, those plugins don't work.
              > I tried that asFFT as well as other xtras that get just the frequency.
              > The frequency isn't the same as the pitch of a voice. I don't see a way
              > to match the frequency from aaFFT to a known key or pitch.

              I'm no musician or sound engineer, but do any of the links in this search
              help?

              http://www.google.com/search?hl=en&q=pitch+key+frequency



              --
              Mark A. Boyd
              Keep-On-Learnin' :)
              • 4. Pitch recognition
                FasterPastor Level 1
                Do the plugins really not work? They don't discern frequency? If they discern frequency, then they do work, and you can translate from frequency to pitch in your code.

                Examples: "A4" refers to the A above middle C (that is, A440), and that note has a frequency of 440.0Hz. Middle C is set as "C4", and that note has a frequency of 261.6Hz.

                The term "scientific pitch notation" will be important to you in such a translation process.

                Wikipedia has a great article about this, including a table (chart) showing the frequency of each pitch. Reading the article (in addition to adopting the table) is highly recommended.

                Edited to add:

                Here's a link to the Wiki article:
                http://en.wikipedia.org/wiki/Scientific_pitch_notation
                • 5. Re: Pitch recognition
                  cratica Level 1
                  Thank you all for your responses. I really appreciate that some are trying to think out loud with me and help me. It's been a bit frustrating! :)

                  I've previously read the article and found the charts concerning frequency. I think the problem lies with what value(s) the aaFFT and other xtras give me. The aaFFT for instance doesn't give me a value anywhere near one of those that are in the charts. They give a group of values that fluctuate depending on how loud you sing into the mic (almost like amplitude). It's like looking at an equilizer. Perhaps the relationship between the values could somehow be determined and that could then be created into some sort of frequency value. Don't know. Think of having 8 values while your are singing in the key of C. 100 50 40 22 15 7 9 12. Now sing in the key of F and you might see something like 100 49 41 21 12 7 9 10. (ficticious numbers). Now sing just a little louder and you have 150 99 92 41 60 49 35 47.

                  That is basically all I can get out of aaFFT. Useless information for what I need. What I would love to have is any true single frequency value such as 4186.01. Then I could say "hey! that's c8!".

                  Question is, how do I get that single frequency value. That value shouldn't change when the amplitude changes as well. (like aaFFT does).

                  Does that make sense? Thanks!!!!
                  • 6. Re: Pitch recognition
                    FasterPastor Level 1
                    I am unfamiliar with any of those plugins. If any one advertises that it discerns frequency, when all it does is discern amplitude, then I suppose it's false advertising in a sense. Such would really not work for your needs here. Sorry I am not more help.
                    • 7. Re: Pitch recognition
                      FasterPastor Level 1
                      I looked on Antoine Schmitt's plugin site, and it looks like it discerns all the frequencies at once, and shows how much amplitude exists at a given frequency, but there does not seem to be much that would decipher from that which of the peaks is to be recognized as the human voice's pitch frequency. I know that they sell tuning devices that listen to the pitch of a guitar string and tell if it needs tuned up or down. Based on that, I'd say it is possible. But I don't know of any product that already does that in Director. I begin to wonder if you might be able to get better advice from talking to someone like Antoine Schmitt directly. If it is possible, maybe he would know more where to start.
                      • 8. Re: Pitch recognition
                        FasterPastor Level 1
                        When I tried to reply to the private message you sent me, the system said you profile indicates you don't want to receive private messages. I just wanted to make sure you knew that setting was chosen in your profile.

                        It is good to make your acquaintance. I pray you have success in your effort. Sorry I am not more help.
                        • 9. Re: Pitch recognition
                          cratica Level 1
                          Exactly. asFFT gives you multiple frequencies and amplitude of those frequencies each sample. Can't really use that information. Just need a simple, single frequency per sample, no amplitude.

                          btw, I modified my profile so I can accept private messages. Thanks for letting me know about that.

                          • 10. Re: Pitch recognition
                            Level 7
                            On Tue, 10 Oct 2006 23:16:49 GMT, cratica posted in
                            macromedia.director.lingo:

                            > Exactly. asFFT gives you multiple frequencies and amplitude of those
                            > frequencies each sample. Can't really use that information. Just need a
                            > simple, single frequency per sample, no amplitude.

                            It would be an exceedingly rare person who can create a perfect single
                            frequency. Even guitar strings and other musical instruments produce
                            haromonic frequencies.

                            I imagine the trick will involve determining which frequency has the highest
                            amplitude (by far) at a given time. Then determine where that frequency
                            resides within the mentioned table.

                            Of course, you'll have to provide tolerances. How often will one produce a
                            perfect 4186.01 pitch? - especially Karaoke singers?



                            --
                            Mark A. Boyd
                            Keep-On-Learnin' :)
                            • 11. Re: Pitch recognition
                              FasterPastor Level 1
                              IAWTC.
                              • 12. Re: Pitch recognition
                                cratica Level 1
                                Of course there will never be a perfect pitch sung for any real length of time. The tolerance will be something I will work on but I'm not concerned about that right now, that's the easy part! :) I have to get some sort of data I can begin to work with and so far I have nothing usable.

                                Amplitude should have nothing to do with it since there are far too many variables with distance to the mic, strength of voice, age, octaves sung in (lower will be far lower amplitude than higher octaves). I need to keep amplitude out of the equation if at all possible. I simply need one value, the pitch/frequency/key... whatever you want to call it.

                                • 13. Re: Pitch recognition
                                  Level 7
                                  cratica wrote:
                                  > Amplitude should have nothing to do with it since there are far too many
                                  > variables with distance to the mic, strength of voice, age, octaves sung in
                                  > (lower will be far lower amplitude than higher octaves). I need to keep
                                  > amplitude out of the equation if at all possible. I simply need one value,
                                  > the pitch/frequency/key... whatever you want to call it.

                                  If you try to sing at a single frequency your voice is producing
                                  multiple frequencies. I think you understood that part already that
                                  there is no special frequency for a voice. Example: If I use my
                                  guitartuner it only recognizes notes that it is expecting but it doesn't
                                  work with a bass guitar very well.
                                  What you could do to analyse the input data is to check what relations
                                  between the input data of fftxtra are near at some sample data. So you
                                  might record values for a very clean note sung and compare that with
                                  your realtime input data to have a decent result. You have to analyse
                                  the frequency spectrum that you get from fftxtra.
                                  • 14. Re: Pitch recognition
                                    duckets Level 1
                                    Perhaps you could use the asFFT xtra to retrieve a small 'sample' portion of the current sound input, and analyse the data within. A very simple method would be to count where the 'zero crossings' occur in the sound file and work out the average distance between the crossings.

                                    You might want to get more complex and smooth out any high frequencies in the data, and have a larger 'dead zone' for your zero crossing code to avoid counting tiny fluctuations around the zero line. Or you could scan for the max and min values, and only count your frequency period where the values reach near these max and min values.

                                    Of course this will only work if the sound input is a simple note, not if there is audible background music or noise.

                                    - Ben
                                    • 15. Re: Pitch recognition
                                      cratica Level 1
                                      what I tried to do with asFFT is sing into the mic and record the values. Then I hardcoded those values in code and created a 'flag' in the case the live input matched the hardcoded values. It sort of worked, but it really isn't accurate enough. There just isn't enough consistency in the values to create any sort of match on a consistent basis. That is only one note! I tried hardcoding two notes and the resolution is just not there using this method.

                                      I was able to contact an old programmer friend of mine I used to work with at a game company and he has solved this exact problem before. I will most likely be hiring him to tackle this issue. We'll see what happens.

                                      Thanks for all the input guys/gals.
                                      • 16. Re: Pitch recognition
                                        duckets Level 1
                                        You shouldn't need any hard-coded data. Just take a sample of the current sound from the mic (using asfft) (1024 bytes, for example).

                                        First scan it for the max and min values.

                                        Then iterate through the values, and make a note each time the values reach near the max value, and then near the min value. You'll need to keep toggling whether you're checking for near the max or min alternately. As you scan the values, you should find your signal oscillates between near your max and near your min a number of times within the whole sample. Each time you encounter a near-max followed by a near-min, note the number of samples between. This is your frequency period. Take the average of all your frequency periods that you find in the sample.

                                        Because the sample rate is 44100hz, you can then calculate the frequency in hz by calculating (44100/averagePeriod).

                                        - Ben