Archive for the ‘Powershell’ Category

Download and delete files from FTP using Powershell

The following Powershell script connects to a FTP server, gets a list of files from a specific subfolder, downloads and deletes them.

It uses the System.URI,system.net.ftpwebrequesT, system.net.WebRequestMethods+, System.Net.NetworkCredential, System.IO.StreamReader and other .Net classes.

$user1='user'
$pass1='password'
$local_target1 = 'D:\997\'
$ftp_uri1='ftp://ftp.myserver.com'
$subfolder1='/outbound/997/'

function GetFilesListAsArray($user,$pass,$local_target,$ftp_uri,$subfolder)
{
 # ftp address from where to download the files
 $ftp_urix = $ftp_uri + $subfolder
 $uri=[system.URI] $ftp_urix

$ftp=[system.net.ftpwebrequest]::Create($uri)

if($user)
 {
 $ftp.Credentials=New-Object System.Net.NetworkCredential($user,$pass)
 }
 #Get a list of files in the current directory.
 #Use ListDirectoryDetails instead if you need date, size and other additional file information.
 $ftp.Method=[system.net.WebRequestMethods+ftp]::ListDirectory
 $ftp.UsePassive=$true

try
 {
 $response=$ftp.GetResponse()
 $strm=$response.GetResponseStream()
 $reader=New-Object System.IO.StreamReader($strm,'UTF-8')
 $list=$reader.ReadToEnd()
 $lines=$list.Split("`n")
 return $lines
 }
 catch{
 $_|fl * -Force
 }
}

function DownloadFile ($sourceuri,$targetpath,$username,$password){
 # Create a FTPWebRequest object to handle the connection to the ftp server
 $ftprequest = [System.Net.FtpWebRequest]::create($sourceuri)

# set the request's network credentials for an authenticated connection
 $ftprequest.Credentials = New-Object System.Net.NetworkCredential($username,$password)

$ftprequest.Method = [System.Net.WebRequestMethods+Ftp]::DownloadFile
 $ftprequest.UseBinary = $true
 $ftprequest.KeepAlive = $false

# send the ftp request to the server
 $ftpresponse = $ftprequest.GetResponse()

# get a download stream from the server response
 $responsestream = $ftpresponse.GetResponseStream()

# create the target file on the local system and the download buffer
 try
 {
 $targetfile = New-Object IO.FileStream ($targetpath,[IO.FileMode]::Create)
 "File created: $targetpath"
 [byte[]]$readbuffer = New-Object byte[] 1024

# loop through the download stream and send the data to the target file
 do{
 $readlength = $responsestream.Read($readbuffer,0,1024)
 $targetfile.Write($readbuffer,0,$readlength)
 }
 while ($readlength -ne 0)

$targetfile.close()
 }
 catch
 {
 $_|fl * -Force
 }

}
function DeleteFile ($file,$username,$password){
 # Delete files from FTP server after download
 $uri2=[system.URI] $file
 $request = [system.Net.FtpWebRequest]::Create($uri2)
 $request.Credentials = New-Object System.Net.NetworkCredential($username,$password)
 $request.Method = [System.Net.WebRequestMethods+FTP]::DeleteFile
 $response = $request.GetResponse()
 "Deleted from FTP: $source" -f $response.StatusDescription
 $response.Close()
}

$lines = GetFilesListAsArray $user1 $pass1 $local_target1 $ftp_uri1 $subfolder1
'List of files OK'
$total_files = 0
foreach ($line in $lines){
 if ($line -ne '') {
 #TRIM the file_name because if it contains blank spaces you will get errors when creating the file
 $file_name = $line.ToString().Trim()
 $source = $ftp_uri1 + $subfolder1 + $file_name
 $target = $local_target1 + $file_name
 DownloadFile $source $target $user1 $pass1
 DeleteFile $source $user1 $pass1
 $total_files = $total_files + 1
 }
}
"Total files downloaded: $total_files"

If you get an error like the following when trying to execute the script:

File C:\scripts\test.ps1 cannot be loaded because the execution of scripts is disabled on this system.
Please see “get-help about_signing” for more details.

Make sure the execution of Powershell scripts is enabled:

Get-ExecutionPolicy

And set it to the appropriate level:

Set-ExecutionPolicy Unrestricted

References:

Running Windows PowerShell Scripts

Using the Set-ExecutionPolicy Cmdlet

(Powershell) Loop to delete files from an FTP Location

Anuncios

This is what I was looking for! PowerShell rocks!!

Ramblings of the Sleepy...

We’re constantly working to standardize how builds get pushed out to our development, UAT, and production servers. The typical ‘order of operations’ includes:

  1. compile the build
  2. backup the existing deployment
  3. copy the new deployment
  4. celebrate

Pretty simple, but with a few moving parts (git push, TeamCity pulls in, compiles, runs deployment procedures, IIS (hopefully) doesn’t explode).

One step to standardize this has been to add these steps into our psake scripts, but that got tiring (and dangerous when we found a flaw).  When in doubt, refactor!

First, get the codez!

DeployTo.ps1 and an example settings.xml file.

Creating a simple deployment tool – DeployTo

The PowerShell file, DeployTo.ps1, should be located in your project, your PATH, or wherever your CI server can find it–I tend to include it in a folder we have that synchronizes to ALL of our build servers automatically via Live Mesh. You could include it with your project…

Ver la entrada original 326 palabras más

Publish ASP.Net web application using MSBuild and PowerShell

PowerShell script to build and publish ASP.Net web application, applying web.config tranformations, using MSBuild.

Considerations:

  • This works for a solution having  7 projects: 1 web application project and 6 related others (DataAccess, BussinessLogic,DomainEntities, Utils, etc).
  • All of these projects run under .Net Framework 3.5.
  • I’m using Visual Studio 2010.
  • Finally I got a MSBUILD error MSB1008 (Only one project can be specified.), but anyway, build and publish worked.

PowerShell script

$build = "$env:windir\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe"
$SolutionPath = "D:\SOLUTION_PATH"
$SolutionFile = "SEGU.sln"
$WebProjectFile = "SEGU\SEGU.csproj"
$WebConfigTransform = "Build-Web.Config.xml"
$OutputPath = "D:\SOLUTION_PATH\PrecompiledWeb\SEGU"
& $build "$SolutionPath\$SolutionFile" /t:rebuild
& $build "$SolutionPath\$WebProjectFile" "/t:ResolveReferences;_CopyWebApplication;publish" /p:OutDir="$OutputPath\bin\" /p:WebProjectOutputDir="$OutputPath"
& $build "$SolutionPath\$WebConfigTransform"

Build-Web.config.xml

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<UsingTask TaskName="TransformXml" AssemblyFile="C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v10.0\Web\Microsoft.Web.Publishing.Tasks.Dll"/>
<Target Name="GenerateConfigs">
<Message Text="Transfom Config." />
<TransformXml Source="SEGU/Web.config" Transform="Web.Release.Config" Destination="PrecompiledWeb/SEGU/Web.config" />
<Message Text="Done Transfom Config." />
</Target>
</Project>

Web.Release.config

<?xml version="1.0"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
<appSettings>
<add key="UserName" value="myuser" xdt:Transform="SetAttributes" xdt:Locator="Match(key)"/>
<add key="Password" value="password" xdt:Transform="SetAttributes" xdt:Locator="Match(key)"/>
<add key="SERVERNAME" value="192.168.10.14" xdt:Transform="SetAttributes" xdt:Locator="Match(key)"/>
<add key="urlServerReporting" value="http://RELEASE_SERVER/ReportServer" xdt:Transform="SetAttributes" xdt:Locator="Match(key)"/>
</appSettings>
<connectionStrings>
<add name="ConSEG" connectionString="Data Source=RELEASE_SERVER;Initial Catalog=Seguros;User ID=username;pwd=password" xdt:Transform="SetAttributes" xdt:Locator="Match(name)"/>
</connectionStrings>
  <system.web>
  <compilation xdt:Transform="RemoveAttributes(debug)" />
  </system.web>
</configuration>

References:

Replicate VS2008 “Publish Web Site” from command line
How do I publish a Asp.net web application using MSBuild?
How to Publish Web with msbuild?
MSBuild Error MSB1008
MSBUILD : error MSB1008: Only one project can be specified. Switch: Education

Web config transformations

Web.config Transformation Syntax for Web Application Project Deployment
How to: Transform Web.config When Deploying a Web Application Project
Transform app.config and web.config
Web.config Transformation Tester
_CopyWebApplication with web.config transformations
Using MSBuild 4.0 web.config Transformation to Generate Any Config File
Config transformations outside of web app builds
Deploying an ASP.NET Web Application to a Hosting Provider using Visual Studio or Visual Web Developer: Web.Config File Transformations – 3 of 12

Powershell: compress backups and FTP transfer

I use this snippet to check my database backups folder for backup files not compressed yet, compress them using 7-Zip, and finally deleting the “*.bak” files to save some disk space. Notice files are ordered by lenght (smallest to biggest) before compression to avoid some files not being compressed.

$bkdir = "E:\BackupsPWS" #backups location directory
$7Zip = 'C:\"Program Files"\7-Zip\7z.exe' #compression utility
$files_to_transfer = New-Object System.Collections.ArrayList #list of zipped files to be transferred over FTP
$ftp_uri="myftpserver"
$user="myftpusername"
$pass="myftppassword"

#find .bak files not zipped yet, zip them, add them to the list to be transferrd
get-childitem -path $bkdir | Sort-Object length |
where { $_.extension -match ".(bak)" -and
-not (test-path ($_.fullname -replace "(bak)", "7z")) } |
foreach {
$zipfilename = ($_.fullname -replace "bak", "7z")
Invoke-Expression "$7Zip a $zipfilename $($_.FullName)"
$files_to_transfer.Add($zipfilename)
}

#find .bak files, if they've been zipped, delete the .bak file
get-childitem -path $bkdir |
where { $_.extension -match ".(bak)" -and
(test-path ($_.fullname -replace "(bak)", "7z")) } |
foreach { del $_.fullname }

#transfer each zipped file over FTP
foreach ($file in $files_to_transfer)
{
$webclient = New-Object System.Net.WebClient
$webclient.Credentials = New-Object System.Net.NetworkCredential($user,$pass) # FTP credentials
$ftp_urix = $ftp_uri + "/" + $file.Substring($bkdir.Length + 1) # ftp address where to transfer the file
$uri=[system.URI] $ftp_urix
$webclient.UploadFile($uri, $file) #transfer the file
}

Powershell: compress files using 7-Zip

This post shows you how to compress a set of files (*.bak) in a directory, using a Powershell script and 7-Zip.

Later, delete the original files and leave only the compressed ones.

Save the script with “.ps1” extension and fire it from a Powershell command line, from a “.bat” file or set it as a Windows scheduled task.

$bkdir = "E:\BackupsPWS"
$7Zip = 'C:\"Program Files"\7-Zip\7z.exe'
get-childitem -path $bkdir | Sort-Object length |
where { $_.extension -match ".(bak)" -and
-not (test-path ($_.fullname -replace "(bak)", "7z")) } |
foreach {
$zipfilename = ($_.fullname -replace "bak", "7z")
Invoke-Expression "$7Zip a $zipfilename $($_.FullName)"
}
get-childitem -path $bkdir |
where { $_.extension -match ".(bak)" -and
(test-path ($_.fullname -replace "(bak)", "7z")) } |
foreach { del $_.fullname }

References:

Compressing SQL Server Backups With Windows PowerShell and 7-zip

Calling 7-Zip from powershell

Powershell: backup and compress SQL databases

The following Powershell script code backups all databases for a specific SQL Server and then compress those files (*.bak) into a ZIP file.

The last line is very important, because it lists the “.bak” files in the directory, ordered by length (ascending), so the compression process doesn’t get interrupted.

Lately, you can call this script from a BAT file and set it up as a scheduled task, so, let’s say, backup all your databases from your server everyday at midnigth, and compress them to a zipped file. Next morning when you arrive to your office, you can download it. Even more, you can schedule it to download it automatically to an FTP server at your data center.

$assemblyInfo = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO")
if ($assemblyInfo.GetName().Version.Major -ge 10)
{
# sql server version is 2008 or later, also load these other assemblies
[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.SMOExtended') | Out-Null
[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.SQLWMIManagement') | Out-Null
}

$s = new-object ("Microsoft.SqlServer.Management.Smo.Server") "(local)" #this can also be "SERVERNAME\INSTANCENAME"

$bkdir = "E:\Backups" #We define the folder path as a variable
$dbstobackup = @(“DB1", “DB2", “DB3")

$dbs = $s.Databases
foreach ($db in $dbs)
{
if($dbstobackup -contains $db.Name)
{
$dbname = $db.Name
$dt = get-date -format yyyyMMddHHmm
$dbBackup = new-object ("Microsoft.SqlServer.Management.Smo.Backup")
$dbBackup.Action = "Database"
$dbBackup.Database = $dbname
$dbBackup.Devices.AddDevice($bkdir + "\" + $dbname + "_db_" + $dt + ".bak", "File")
$dbBackup.SqlBackup($s)
}
}

# Create a new zip file from pipeline
function Create-Zip()
{
param
(
[string]$zipFile
);

New-Zip -zipfileName $zipFile
$zip = Get-Zip -zipfileName $zipFile

#loop through files in pipeline
foreach($file in $input)
{
#add file to zip and sleep 1/2 second
$zip.CopyHere($file.FullName)
Start-sleep -milliseconds 15000
}

}

#create a new zip file
function New-Zip
{
param([string]$zipfilename)
set-content $zipfilename ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
(dir $zipfilename).IsReadOnly = $false
}

#get the zip file
function Get-Zip
{
param([string]$zipfilename)
if(test-path($zipfilename))
{
$shellApplication = new-object -com shell.application
$zipFullFilename = (get-childitem $zipfilename).FullName
$shellApplication.NameSpace($zipFullFilename)
}
}

$dtzip = get-date -format yyyyMMddHHmm
$zipfilename = $bkdir + "\DB_Backups_" + $dtzip + ".zip"
Get-ChildItem -path $bkdir -filter *.bak* | Sort-Object length | create-zip $zipfilename

References:

Backup SQL Server Databases with a Windows PowerShell Script

Loading SMO Assemblies into PowerShell

PowerShell Tips