cowboyrushforth.com

dspam experiement

Fri, 31 Oct 2008 13:56:00 -0700

So finally, SpamAssasin just could not hack it any more on my server. This year there has been a steady influx of spam, and some users of my mail services are literally getting 1 good message out of 100 messages. I feel like I have truly exhausted all SA resources out there. From custom rule sets after more rule sets, after more configuration, SA just can't seem to learn fast enough. I also employ RBL/SBL checks and the works. So, what to do? Enter DSPAM.

Previously, I was a bit weary when I heard about the DSpam project and at the time, SpamAssassin was working for me, so why fix something that is not broken or so I thought. Well that mindset finally passed, and I dove into configuring DSpam for my setup. I run a Postfix/Cyrus type of a virtual email setup that is pretty complicated to say the least. I knew in advance that embarking down this road would mean learning some things and some elbow grease, but that is ok. I finally settled on a configuration that is similar to both Neale's Setup and Cepcep's Setup in different ways. In my setup, I have 2 independent chains, one for inbound email and one for outbound email. I decided to leave my outbound chain alone, and continue to send it thru amavisd. For the inbound chain, all mail is subject to getting sent to dspam, then dspam re-routes it back into postfix with a result attached.

Lastly, I wrote the following script, so that I may simply forward my mails to ham@ or spam@, and have postfix deliver it directly to my script, for retraining purposes.

It is piped to dspam, from postfix, like so:

dspam-retrain   unix    -       n       n       -       10      pipe
  flags=Ru user=dspam argv=ruby /usr/local/bin/dspam-retrain.rb $nexthop $sender $recipient

#!/usr/bin/ruby
# Ruby version of dspam-retrain perl script.
# Perl version: http://dspamwiki.expass.de/DspamRetrainScript
# Author: wsr@rushforthnetworks.com
# License: BSD
# Abstract: setup postfix to pipe mail into this script
#           so that we may then pass it off to dspam in
#           an appropriate way.
#
#           dspam-retrain.rb handles spam-user@domain.tld, and
#                                    spam@domain.tld formats
#
#           If dspam-retrain.rb does not find a signature,
#           it will train dspam using corpus or innoculation
#           mode.
#
# Requirements: open4 gem is installed
# -----------------------------------------------------------

#### Configuration BEGIN ####################################

#enable logging?

@enable_logging = true
@log_file = '/tmp/dspam_retrain.log'

#what mode to train dspam in if we do not find signature?
#corpus or innoculation

@alternative_mode = 'corpus'

#### Configuration END #####################################

require 'rubygems'
require 'open4'

if @enable_logging
        require 'logger'
        @logfile = File.new(@log_file, 'a+')
        @log = Logger.new(@logfile)
end

def logthis(message)
        if @enable_logging
                @log.info(message)
        end
end

# Get arguments
spam_class  = ARGV[0]
sender = ARGV[1]
recip  = ARGV[2]

logthis("dspam-retrain Started. Arguments: #{spam_class}, #{sender}, #{recip}")

#see if we were passed spam-user@ or just user@
if match = recip.to_s.match(/^(spam|ham)-(\w+)@/)
        user = recip.gsub(/#{match[1]}\-/, '')
elsif match = recip.to_s.match(/^(\w+)@/)
        user = sender
else
        logthis("\tCant't determine user")
        exit 75
end

signature = String.new
message   = String.new

#loop through email (passed via stdinput)
#search for signature

$stdin.each do |line|
        if line.match(/X-DSPAM-Signature/)
                signature = line.gsub(/X-DSPAM-Signature:/, '')
                #remove any potential whitespace
                signature.strip!
                #since we found signature, break loop
                break
        end
        message << line
end

if signature.length.to_i == 0
        #we did not find a signature, do normal training

        mode = 'train'

        logthis("\tEmail did not have signature passed in. Attempting #{@alternative_mode} train.")

        #open up dspam with appropriate options
        pid, dspam_in, dspam_out, dspam_err = Open4::popen4 "/usr/bin/dspam --source=#{@alternative_mode} --class=#{spam_class} --user #{user}"

        #attempt to feed message in
        begin
                dspam_in << message
        rescue
                #means dspam closed stdinput because our
                #options failed
                dspam_err.each_line do |o|
                        logthis("\t" + o.gsub(/\n/, ''))
                end
        end
        #close dspams stdinput
        dspam_in.close_write

        #see if dspam left any messages for us
        dspam_out.each_line do |o|
                if o.strip.length == 0 then next end
                logthis("\t" + o.gsub(/\n/, ''))
        end

else
        #we found signature, so we will only pass that.

        mode = 'retrain'

        logthis("\tRetraining Signature: #{signature} for User: #{user} as: #{spam_class}")

        #open up dspam with appropriate options
        pid, dspam_in, dspam_out, dspam_err = Open4::popen4 "/usr/bin/dspam --source=error --signature=#{signature} --class=#{spam_class} --user #{user}"

        #see if dspam left any messages for us
        dspam_out.each_line do |o|
                if o.strip.length == 0 then next end
                logthis("\t" + o.gsub(/\n/,''))
        end
        dspam_err.each_line do |o|
                if o.strip.length == 0 then next end
                logthis("\t" + o.gsub(/\n/,''))
        end
end

ignored, status = Process::waitpid2 pid

#see if we exited cleanly and log it
if status.exitstatus == 0
        logthis("\tMessage successfully #{mode}ed as #{spam_class}")
else
        logthis("\tMessage NOT #{mode}ed")
end

back from the dunes

Wed, 15 Oct 2008 20:27:00 -0700

I have arrived back from the dunes, mostly in one piece. It was an amazing trip, and one that will surely be repeated hopefully soon!

We rode for 3 days solid, riding for almost the whole day each day, and resting at night. The terrain was very diverse, and we were able to ride right out of our campground and go for miles and miles. About half of the time it was raining, which made for very cold nights, and very hard packed sand.

I managed to cartwheel my yz250, tearing off some fenders, and my better half, managed to blow up her freshly rebuilt engine on her cr125. Somehow in the mess of it all it was one of the best vacations we have taken in a bit. Lucky for me I was able to score some makeshift parts for free from a local bike shop in Coos Bay, Oregon. The wife's bike is still out of commission, but I have already started to tear it down, so it will be roaring back to life soon.

Click here for more photos.

dunes and decompression

Tue, 07 Oct 2008 00:07:00 -0700

It is that time of the the year, that the dunes are open and calling my name. Well actually they call my name every year all the time, but this it the first year in a while I will be able to make my families annual camping trip to the Oregon coast to ride the dunes.

We are going to a spot that is a few minutes away from Coos Bay. I am not sure I have been to this exact spot before, but I am pretty stoked. The bikes are clean, lubed, and armed with paddle tires, the fleet is ready. Hopefully I will have time to snap a few pictures, cross your fingers.

After wrangling the sand for a few days, we will be heading out just in time to hit San Francisco Decompression, where we will see a whole slew of bats, and other neighbors from home. Not too long after this, I will also be attending San Diego Decompression, too, making my decompression circuit for this year probably complete.

Happy Trails
-spotman

energy and momentum

Wed, 01 Oct 2008 01:08:00 -0700

I have been thinking a lot about energy lately. In my quest to stay on top of the constant barrage of work, I have exhausted, and employ, all normal resources for maintaining a good energy and efficiency level, including all sorts of drinks, kombucha, tea, sleep, exercise, etc. As I dig deeper into my routine, I am finding that while caffeine and sleep are vital, it's really just the tip of the iceberg.

By routine, I guess this is the most robotic I have ever been. Wakeup, Coffee, Code, Coffee, Code, Lunch, Code, Dinne, Code, Sleep. Wash, Rinse, Repeat. For the past couple years that has been just about it, except I do get out in the evenings and afternoons, and the occasional vacation or camping trip, but definitely not an enormous amount.

Don't get me wrong, I love my job, its awesome - but it is hard to be "on" everyday. Somedays are very long, and as much as you want to try and really tackle the 38 tasks in front of you, something keeps you from doing it. It might be an email, or a phone call, or nothing at all. Just knowing that in one day you can't make a dent, it keeps you from moving very quickly at all.

Other days are a little bit better, but even more tiring when they are done. You have got your task list down to about 20 things, so you give it all you got, hoping to at least cut it in half. However, somehow, you hit a huge patch of mud. You are stuck in it all day. A bug that should take 15 minutes to fix turns into a 4 hour issue and ends up being 1 line that you stared at for 2 of those hours. Client decides that they want a magic lever, attached to a ceiling, with custom elevator shoes to reach it. At the end of the day you have worked very hard, but only finished 3 things.

Then there are days where you somehow find it. It, being the drive inside, energy even, to keep you going all day long, just knocking off things from your list. Today was one of those days for me. It is definitely an ebb and flow, but what is more interesting than having a flow day, is that it seems like everything just flies by, effortlessly. A 15 minute bugfix, takes 15 minutes. Clients requests are sane. All the sudden, I am right where I need to be. It is such a huge contrast from the other scenarios, but it certainly seems like it really is all about momentum at times. Possibly because I just spent the weekend seeing a close friend get married. There was definitely a lot of good energy there.

Cheers to energy and momentum. Have a nice week.

Tech Toy Review

Thu, 18 Sep 2008 20:26:00 -0700

Tech Toy Review has just been launched! Tech toy review is mostly ran by a friend of mine, but I will also be contributing the occasional review. So stay tuned, and make sure to check it out :)

The G-Spot

Thu, 18 Sep 2008 18:29:00 -0700

Enter the G-Spot. A drink I invented just now. If there is already a drink with this name (i have not checked), it should be replaced with this recipie:

35% Pama Liqueur
20% Dark Rum
30% Guava Juice/Nectar
15% Tonic Water

The bigger the glass the better. On the rocks. Stirred, or shaken, seems about right either way; enjoy.

-spotman

Living the Dream

Mon, 08 Sep 2008 20:53:00 -0700

It has been the most amazing Burning Man ever. I am finally starting to gain feeling back in my limbs and pick up speed in the real world. Often with a lot of experiences one goes through in life, the first time you experience something, you get a feeling that is very hard to achieve again. Whether its white water rafting, or the first time you win a race, you had never known what it was going to be like previously, so you went into it full bore, and came out victorious. How does one achieve this feeling again? Until Burning Man 2008, I did not believe it was possible.

This year was incredible, and to be perfectly honest, there are no words which can express the depth of energy that this year contained. Bat Country was explosive, and infectious. Every single member of our camp went the extra mile this year. We asked for people to step up and bring extra supplies, supplies were there. We asked people to step up and take extra trash home, it was taken. We had the most stress free, successful week of our lives known to date, and for just a few nights at the end of the week, after we had worked very hard to build our camp and meet our original goals, Bat Country, was the place to be on the playa, at least for almost every single bat. We rocked the playa and fully satisfied our camp's goals. We are now even more motivated and want to bring the same theme back in 2009, and be ready for Evolution.

There was so much energy pumped into our camp this year due to all the awesome people that are in it, it was completely awesome and unbelievable. Thank you to everyone out there in Bat Country that helped make this year a success. The man burns in 362 days. Bat Country will be ready.

Few pictures here. more coming soon.

freedom glory burn

Wed, 03 Sep 2008 10:26:00 -0700

We went, we conquered, we destroyed, we laughed, and we cried. Burning Man 2008, is now officially over.

We spent a huge portion of the time working on our 18 foot in diameter spinning carousel bar, but with absolutely no stress, and huge insane success. Before the weekend arrived we had it completed, and we were spinning, in all senses of the word.

Everybody in Bat Country stepped up this year, and to me, this was my favorite year so far, and my head is overloaded and still spinning from all the intense energy.

What goes up must come down, and so far re-entry has been harder than ever for me this year. I think I am now well past the half way mark emotionally, but the first few hours, from the exit gates, to wadsworth, were filled with tears and emotion. I realized before this I had not cried for 2 entire years.

If it is even possible, this year was more powerful and awesome than my virgin year, which was also a completely overwhelming and perfect experience.

Our days were filled with wonderful people, awesome events, and beautiful neighbors. We are really hoping to keep our Esplanade placement, and only increase our game next year.

So for now, spotman out. Media coming very soon I hope.

36 hours remaining

Wed, 20 Aug 2008 08:17:00 -0700

Well, it doesn't get much closer than this. I leave for the playa in 36 quick hours. Things have come a long way in the past couple weeks. My camp's numbers have swelled quite a bit, which brings new energy and new people. Everyone in the camp is super positive and super energized, the buzz is completely overwhelming, and awesome. We have nearly finished every project that we as a camp set out to do, complete from a spinning bar (18' in diameter!), to first class dj's and a sound system, and several major pieces of art!

I believe I have just about everything packed, from the lizard tail, to the solar lights, and we are foaming at the mouth. So this will probably be my last post here on this side, but I will definitely see you on the other side! Sometime after the 3rd of september.
Cacaaaaaw!

If you make it to the playa this year, make sure and swing by 9:30 and Esplanade to say hello!

esplanade countdown

Wed, 30 Jul 2008 23:22:00 -0700

Well, Burning Man just released camp placements this week, and believe it or not, my camp has landed on the ESPLANADE!.

So, this ultimately means that Bat Country will be on what is essentially Main St. Burning Man. On the esplanade we will have the opportunity to expose our art and our camp to many many more people, and in exchange we will have a healthy dose of non-stop action for the entire time of the event.

Needless to say, I am pretty excited. This will be my third year straight attending the burn, and it is unreal to me how it just gets more exciting each year. Each year so far has presented new challenges, adventures, and memories, but this year by far has surpassed the previous two in terms of challenges. Adventures and Memories pending.

I believe Bat Country consists of some of the finest people in the world, and in turn will make this one for the books. From a spinning carousel bar, to Hunter S Tuesday, coming to a playa, near you!

Click For Flag Details