Showing posts with label variables. Show all posts
Showing posts with label variables. Show all posts

Tuesday, March 27, 2012

ability to use variables in package configurations and set their values at runtime

Hi,

My scenario:

I am using a FTP Connection Manager and the configuration setting for it is being set in the package configuration xml file. So the xml file contains the Ftpserver, FTp server username and password. The package is picking up the values from the xml file and is executing successfully. I have to do this because I was not able to provide an expression to the Connection Manager Server Password property.

Now, I want to pick up the ftp details from a database table and set it in the xml file during runtime. Is this possible? OR something like using the

<Configuration ConfiguredType="Property" Path="\Package.Connections[FTP Connection Manager].Properties[ServerPassword]" ValueType="Variable"><ConfiguredValue>@.[user::FtpPassword]</ConfiguredValue></Configuration>

Kindly look at the items in bold. Is this possible? Then I can set the value of the variable in the package before the FTP connection manager task is executed.

Thanks for all the help.

$wapnil

You could use a configuration storage as DB.

Or you can use Script task to query the DB and set the FTP connection properties or Variable values at runtime

Or even Execute SQL task to get the variable values from DB|||

Thanks

Reply to the options that you have provided

1) We want to use XML for the configuration. Just something which we want to follow if we could.

2) We are trying to minimuize the script tasks in the package by bringing the configuration details outside of the package and storing it in XML.

3) same as 1

Lets take a FTP connection Manager for example. The details of the FTP server are present in the database and I have also created variables for that but atleast I am not able to set the FTP Server password using a variable so I resorted to using the xml file and putting in the password there.

Now if I want to connect to multiple FTP server using the same package and the connection details are there in the database then I can pull out the connection details using an Execute SQL task but then how to update the xml files with values......knowing that the FTP server password can be only be set dynamically using the xml file.

Correct me if I am wrong. is storing the variable in the DB the only solution.

Thanks,

$wapnil

|||

spattewar wrote:

Now if I want to connect to multiple FTP server using the same package and the connection details

Are you trying to connect multiple FTP servers at the same time or are you trying to run the package in different environments?

For example if I would need to pull info from 3 different FTP servers - I would create 3 different configuration files:
1 for Development
1 for staging
and 1 for Production site|||

I am trying to connect to multiple FTP servers. But it can be at the same time or in sequence. But I am planning to use a single package for this task.

Thanks for your response.

$wapnil

|||Why don't you use Parent Package Configurations instead? A simple 2-package setup could work where:
Your mother package cycles through a list of FTP server configurations or whatever is needed.|||

That looks like a good idea.

But there is one concern. Even though I pass the configuration as a parent package variable to the child package, will I be able to set the FTP ServerPassword property of the FTP connection manager using a variable. I doubt I can do that, then it boils down to the same thing picking up the package configuration from the xml file or a database, here we would prefer xml file.

Maybe we can do this by.

1) Running the master package which will have a script task to update a xml configuration file.

2) Pass that xml file as a configuration parameter to the child package and execute the child package.

Thanks for your response.

$wapnil

|||I don't have any reason to believe that you can't pass the password or even the whole connectionstring as a variable.

However do note that if your variable values refuse to change (although I don't see any reason why they won't) you might have to tweak the ProtectionLevel of the package to DontSaveSensitive or some other value that allows you to modify your connectionstring variable.

HTHsql

Tuesday, March 20, 2012

A truncation occurred during evaluation of the expression

First of all, I get the following error message for one of my packages which uses user variables:

SSIS package "UsageAnalysis.dtsx" starting.
Information: 0x4004300A at Perform xmlState Shredding, DTS.Pipeline: Validation phase is beginning.
Information: 0x4004300A at Update Analysis Table, DTS.Pipeline: Validation phase is beginning.
Information: 0x4004300A at Update Analysis Table, DTS.Pipeline: Validation phase is beginning.
Error: 0xC001700E at UsageAnalysis:
A truncation occurred during evaluation of the expression.
Error: 0xC0019004 at UsageAnalysis: The expression for variable "GetAnalysisData" failed evaluation. There was an error in the expression.
Error: 0xC02020E9 at Update Analysis Table, UsageAnalysis Source [1]: Accessing variable "User::GetAnalysisData" failed with error code 0xC001700E.
Error: 0xC0024107 at Update Analysis Table: There were errors during task validation.
Warning: 0x80019002 at Usage Analysis Process: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (5) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
SSIS package "UsageAnalysis.dtsx" finished: Failure.

Now my package has the following variables:

GetMaxUsageID: scope package level, type string, statement SELECT MAX(UsageID) AS MaxUsageID FROM XX.XXX

MaxUsageID: scope package level, type int32, default value 0, value get assigned from the following statement executed from sql task that runs GetMaxUsageID variable as above

GetAnalysisData: scope package level, type string, Evaluate as Expression

"SELECT * FROM dbo.UsageAnalysis WHERE UsageID > " + (DT_STR, 8, 1252) @.[User::MaxUsageID]

The package has worked fine until MaxUsageID value reached to 10,00,000 and since then I have been getting above mentioned error message. The problematic step is related to Data Flow task where I use GetAnalysisData. I have tried replacing user variable with literal as follows

"SELECT * FROM dbo.UsageAnalysis WHERE UsageID > 1000000"

the error message stays the same. Please note that package has worked fine before and it still works ok if I don't use user variables. Obviously, some of you would see eliminating user variables as workaround but I would appreciate if cause of that error message could be investigated.

Thanks,

Asaf

Are you sure MAX(UsageID) is 1,000,000 and not 100,000,000?

Try making your cast a longer length in the GetAnalysisData variable. Perhaps: (DT_STR,20,1252)@.[User::MaxUsageID]

|||

Hi Phil,

Thanks for your reply but I am afraid I already have tried the solution you suggested. Max(UsageID) is calculated against sql table and then stored to MaxUsageID user variable.

select max(usageID) from dbo.XXX

--1027005

So you could see value by no means is tens of millions as you anticipated. I also have tried changing DT_STR string length parameter to larger values as proposed

SELECT * FROM dbo.UsageAnalysis WHERE UsageID > " + (DT_STR, 20, 1252) @.[User::MaxUsageID]

Even when I have no casting involved such as

"SELECT * FROM dbo.UsageAnalysis WHERE UsageID > 1000000"

I still get the same truncation error message. I would appreciate if you could suggest something else which obviously I am missing here.

Thanks,

Asaf

|||

Phil,

I have worked it out.

It looks like one of the SSIS bugs. Even though I updated the expression by changing (DT_STR, 6, 1252) to (DT_STR, 12, 1252) so that I could avoid any obvious truncation issues for values above 1 million and after saving the package I rebuilt it but project would fail whenever I executed it.

When I opened the project again, I ended up having the old value for (DT_STR, 6, 1252) in xml code view. Somehow SSIS wouldn't save the updated expression. Hence I kept getting that error message even though I had reasons to believe I had done everything right.

To resolve that, I just decided to define a new variable with required DT_STR string length and my package works without any complain.

Thanks,

Asaf

|||

Sounds like you may have been trying to update the "Value" parameter, not the "Expression" parameter. In that case, it would revert back to whatever you had in the expression.

With that being said, if it is a bug and you can reproduce it in the future, please submit it at http://connect.microsoft.com/sqlserver/feedback and report back here with the link to your submission. But please only submit it if you can reproduce it.

Thanks,

Phil

sql

A truncation occurred during evaluation of the expression

First of all, I get the following error message for one of my packages which uses user variables:

SSIS package "UsageAnalysis.dtsx" starting.
Information: 0x4004300A at Perform xmlState Shredding, DTS.Pipeline: Validation phase is beginning.
Information: 0x4004300A at Update Analysis Table, DTS.Pipeline: Validation phase is beginning.
Information: 0x4004300A at Update Analysis Table, DTS.Pipeline: Validation phase is beginning.
Error: 0xC001700E at UsageAnalysis:
A truncation occurred during evaluation of the expression.
Error: 0xC0019004 at UsageAnalysis: The expression for variable "GetAnalysisData" failed evaluation. There was an error in the expression.
Error: 0xC02020E9 at Update Analysis Table, UsageAnalysis Source [1]: Accessing variable "User::GetAnalysisData" failed with error code 0xC001700E.
Error: 0xC0024107 at Update Analysis Table: There were errors during task validation.
Warning: 0x80019002 at Usage Analysis Process: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (5) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
SSIS package "UsageAnalysis.dtsx" finished: Failure.

Now my package has the following variables:

GetMaxUsageID: scope package level, type string, statement SELECT MAX(UsageID) AS MaxUsageID FROM XX.XXX

MaxUsageID: scope package level, type int32, default value 0, value get assigned from the following statement executed from sql task that runs GetMaxUsageID variable as above

GetAnalysisData: scope package level, type string, Evaluate as Expression

"SELECT * FROM dbo.UsageAnalysis WHERE UsageID > " + (DT_STR, 8, 1252) @.[User::MaxUsageID]

The package has worked fine until MaxUsageID value reached to 10,00,000 and since then I have been getting above mentioned error message. The problematic step is related to Data Flow task where I use GetAnalysisData. I have tried replacing user variable with literal as follows

"SELECT * FROM dbo.UsageAnalysis WHERE UsageID > 1000000"

the error message stays the same. Please note that package has worked fine before and it still works ok if I don't use user variables. Obviously, some of you would see eliminating user variables as workaround but I would appreciate if cause of that error message could be investigated.

Thanks,

Asaf

Are you sure MAX(UsageID) is 1,000,000 and not 100,000,000?

Try making your cast a longer length in the GetAnalysisData variable. Perhaps: (DT_STR,20,1252)@.[User::MaxUsageID]

|||

Hi Phil,

Thanks for your reply but I am afraid I already have tried the solution you suggested. Max(UsageID) is calculated against sql table and then stored to MaxUsageID user variable.

select max(usageID) from dbo.XXX

--1027005

So you could see value by no means is tens of millions as you anticipated. I also have tried changing DT_STR string length parameter to larger values as proposed

SELECT * FROM dbo.UsageAnalysis WHERE UsageID > " + (DT_STR, 20, 1252) @.[User::MaxUsageID]

Even when I have no casting involved such as

"SELECT * FROM dbo.UsageAnalysis WHERE UsageID > 1000000"

I still get the same truncation error message. I would appreciate if you could suggest something else which obviously I am missing here.

Thanks,

Asaf

|||

Phil,

I have worked it out.

It looks like one of the SSIS bugs. Even though I updated the expression by changing (DT_STR, 6, 1252) to (DT_STR, 12, 1252) so that I could avoid any obvious truncation issues for values above 1 million and after saving the package I rebuilt it but project would fail whenever I executed it.

When I opened the project again, I ended up having the old value for (DT_STR, 6, 1252) in xml code view. Somehow SSIS wouldn't save the updated expression. Hence I kept getting that error message even though I had reasons to believe I had done everything right.

To resolve that, I just decided to define a new variable with required DT_STR string length and my package works without any complain.

Thanks,

Asaf

|||

Sounds like you may have been trying to update the "Value" parameter, not the "Expression" parameter. In that case, it would revert back to whatever you had in the expression.

With that being said, if it is a bug and you can reproduce it in the future, please submit it at http://connect.microsoft.com/sqlserver/feedback and report back here with the link to your submission. But please only submit it if you can reproduce it.

Thanks,

Phil

Thursday, February 9, 2012

A few things to do with store variables.

Okay,

I'm creating a small application for my local school district that will allow them to do a small management of inventory.

    Is there a way to create an executable script that will create the default databases the first time? If so, what all do I need to do for the script and to make it execute? I want to make a section in my OPTIONS area where users can put the path to their db (will not be hosted on the same PC as the software) along with the username and password for the db. I know I can store this information in a predetermined variable in the program so that it can reference it off and on.

Could someone please help me with this..... If it's in the wrong forum please let me know.

Thanks,

QWERTYtech

What kind of database does your program use? Full SQL Server, SQL Server Express, Access, or other?|||SQL Express|||

Move the thread from Visual Basic General. There are more exports on storing variables in SQL Express database. I believe you can get satisfying answers in this forum.

Thanks again for your question.

|||

Perhaps these sources will help:

SQL Server 2005 UnAttended Installations
http://msdn2.microsoft.com/en-us/library/ms144259.aspx
http://msdn2.microsoft.com/en-us/library/bb264562.aspx
http://www.devx.com/dbzone/Article/31648